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Supervisor:  G.  Edward  Gibson,  Jr. 

For  years  the  construction  industry  has  struggled  with  hiring  qualified 
Architect  and  Engineering  (A/E)  firms.  The  question  proposed  today  is  if  the 
benefits  derived  from  going  through  a  long,  laborious  process  worth  the  added 
time  and  money  spent?  The  United  States  deemed  this  concept  so  crucial  that  the 
Brooks  Act  was  implemented  in  1972.  Essentially,  the  Act  mandates  that  all 
design  work  must  be  awarded  based  on  qualifications  of  the  A/E  firms  rather  than 
price.  In  today’s  construction  industry,  the  private  sector  has  begin  to  apply  some 
price  pressures  on  A/E  firms.  This  study  will  focus  on  comparing  the  overall 
success  of  office  building  projects  that  have  used  purely  the  qualifications  versus 
projects  that  have  used  some  price  considerations  in  the  selection  process  of  their 
A/E  firms. 
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Chapter  1:  Introduction 


1.1  Purpose 

For  over  25  years  the  Government  and  many  in  of  the  private  construction 
industry  have  been  using  the  Brooks  Act,  a  qualification-based  selection  (QBS) 
process,  to  hire  A/E  services  for  construction  projects.  The  consensus  firom  many 
design  professionals  is  that  the  quality  of  a  project  is  directly  related  to  the  method 
in  which  the  firm  is  selected.  Although  this  feeling  is  shared  by  many  A/E 
designers,  there  has  never  been  any  significant  data  gathered  to  confirm  the 
benefits  of  the  Brooks  Act.  The  Act  was  adopted  in  the  1970’s  not  only  by 
federal  agencies  but  also  by  all  50  states.  Since  then  three  states,  Florida, 
Maryland  and  Massachusetts,  have  changed  their  laws  allowing  owners  to  choose 
design  services  based  on  price  alone.  Maryland  has  since  reverted  back  to  QBS. 
(Vanden  Bosch  et  al.  1996) 

According  to  the  laws  in  most  states,  the  sole  method  of  contracting  with 
an  A/E  firms  for  public  projects  should  be  QBS.  However,  reports  from  within  the 
construction  industry  have  shown  a  significant  trend  that  many  owners  are  using 
price  to  determine  selection  of  A/E  services.  This  brings  up  two  key  questions; 
What  degree  of  difficulty  and  how  successful  are  these  projects?,  and  how  much 
did  price  control  the  selection  of  the  design  services?  The  Construction  Industry 
Institute  has  done  a  preliminary  study  that  set  up  the  framework  for  an  intensive 
study  to  be  done  in  this  area.  (Vanden  Bosch  et  al.  1996) 

Among  the  many  issues  this  study  addressed,  are  the  various  types  of  A/E 
selection  methods.  From  this  study.  Table  1.1  gives  5  methods  of  A/E  selection. 
The  first  three  methods  are  categorized  as  “price-based”  selection  and  the  last  two 
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are  QBS.  As  project  data  are  analyzed  using  the  Questionnaires  generated  by 
phase  I  of  the  Cn  investigation,  the  projects  will  be  separated  into  the  two 
categories.  This  will  allow  a  thorough  analysis  to  be  conducted  on  QBS  versus 
non-QBS  projects.  The  final  goal  using  the  data  obtained  from  randomly  selected 
projects  is  to  determine  whether  the  QBS  process  provides  any  significant  impact 
to  the  overall  success  of  the  projects. 


Table  1 . 1  A/E  Selection  Methods  (Vanden  Bosch  et  al.  1 996) 


DESCRIPTION 

Non-OBS 

Bid 

(Price) 

Price  is  the  only  selection  determinate. 

Combination 

(Pre-qualification,  Price) 

Two 

envelope 

The  first  envelope  contains  qualifications  only.  All  highly 
qualified  firms  are  pre-qualified.  Pre-qualified  firms  are  invited 
to  submit  price  proposals.  The  second  envelope  contains  the 
price  proposal.  Award  is  based  on  best  price. 

One 

envelope 

(Best  value,  considering  qualifications  and  price) 

Firms  simultaneously  submit  qualifications  and  price.  Award 
may  be  based  on  some  combination  of  price  and  qualifications. 

QBS 

Two 

envelope 

The  first  envelope  contains  qualifications  only.  The  one  best 
qualified  firm  is  selected  and  asked  to  submit  a  price  proposal. 
Price  is  negotiated  with  the  one  best  qualified  firm  only. 

Sole  source 

(Reputation  is  the  selection  determinant) 

The  firm  is  selected  based  on  their  reputation  and/or  familiarity 
with  client’s  project.  Price  may  or  may  not  be  negotiated. 

1.2  Objectives 

Naval  Facilities  Command  (NAVFAC)  spends  millions  of  dollars  a  year 
contracting  private  A/E  sources.  The  Phase  I  report  has  discovered  a  growing 


trend  in  the  private  sector  to  consider  price  more  of  a  factor  during  the  A/E 
selection  process.  The  primary  focus  of  this  study  is  to  determine  if  there  is  any 
benefit  to  these  new  selection  processes.  Is  the  construction  industry  saving 
money  and  getting  the  same  quality  facility?  One  of  NAVFAC’s  goals  is  to 
continually  search  for  better  ways  to  provide  support  to  the  Fleet.  This  study  will 
provide  unbiased  information  and  possibly  a  way  for  NAVFAC  to  provide  A/E 
services  faster  and  cheaper  to  their  customers.  The  four  objectives  for  this  study 
are: 

1)  Firmly  establish  NAVFAC’s  position  with  respect  to  the  Brooks  Act. 

2)  Analyze  randomly  selected  project  data  to  determine  if  QBS  versus 

non-QBS  of  design  service  have  an  impact  on  final  outcome  of  a 

construction  project. 

3)  Critique  the  questionnaire  developed  by  CII. 

4)  Outline  possible  policy  implications  for  NAVFAC  based  on  this  study. 

1.3  Structure  of  Thesis 

The  following  chapters  will  present  background  information  on  the  Navy’s 
current  policy  with  respect  to  the  Brooks  Act  and  CII’s  Phase  I  report  which 
includes  a  thorough  literature  review.  Once  the  background  has  been  established, 
the  remaining  chapters  will  include  the  research  methodology,  presentation  and 
data  analysis  of  a  pilot  study,  as  well  as  conclusions  and  policy  implications  for 
NAVFAC. 
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Chapter  2  Background 


2.1  Introduction  to  Navy  A/E  Contracting 

Over  the  past  decade,  there  has  been  a  move  to  improve  methods  in 
contracting  for  facility  design.  The  emphasis  has  been  placed  on  providing  better 
services  without  compromising  quality  of  design  and  construction.  This  concept 
coupled  with  the  draw  down  of  the  military  has  forced  the  Government  to  look  at 
new  and  innovative  methods  in  the  contracting  arena.  One  of  the  areas  under 
scrutiny  is  outsourcing  of  design  to  A/E  firms. 

There  appears  to  be  a  transition  phase  in  effect  within  NAVFAC  with  an 
increased  use  of  Design/Build  projects.  One  study  revealed  Design/Build  projects 
on  child  care  facilities  realized  a  $20  per  square  foot  (SF),  75  percent  reduction  in 
the  number  of  design  changes,  and  completed  the  projects  8  months  earlier  than 
the  typical  Design/Bid/Build  projects.  (Roth  1995)  These  are  significant  results, 
although  the  study  was  performed  on  a  small  sample  of  projects.  Not  only  does 
the  owner  get  the  use  of  the  facility  earlier  at  less  cost,  but  also  the  contracting 
administration  team  can  focus  on  other  projects.  This  reinforces  the  concept  of 
working  more  efficiently  with  less  work  force. 

If  Design/Build  contracting  seems  so  appealing,  then  why  is  the  transition 
in  using  this  process  occurring  at  a  relatively  slow  pace?  Naval  Facilities 
Engineering  Command  (NAVFACENGCOM)  consists  of  thousands  of  people 
such  as  engineers,  lawyers,  contracting  officers,  contract  administrators  and 
inspectors  who  all  play  a  major  role  and  work  in  different  offices  around  the  world. 
Not  only  must  thousands  of  people  be  trained  to  administer  Design/Build 


4 


contracts,  but  also  the  legal  aspects  need  to  be  reviewed  so  that  free  and  open 
competition  is  not  compromised.  In  addition,  the  advantages  and  disadvantages 
must  be  analyzed  to  help  in  determining  if  Design/Build  is  a  viable  long  term 
solution.  Tables  2.1  and  2.2  summarize  some  of  the  advantages  and  disadvantages 
associated  with  Design/Build  contracting.  (Roth  1995) 


Table  2.1  Design/Build  Advantages  (Roth  1995) 


Area  of 
Impact 

Design/Build  Advantages 

A  Time 

•  Use  of  fast-track  concepts  allows  project  to  be  completed  more 
quickly. 

•  Project  can  be  prepared  for  solicitation  and  awarded  quickly. 

•  Design/Build  has  been  proven  to  be  30%  faster  at  delivering  the 
project  in  some  studies. 

Cost 

•  Guaranteed  maximum  price  is  established  early  in  the  process. 

•  Number  of  modifications  significantly  reduced. 

•  In-house  staff  can  be  effectively  used  for  IFB  development. 

•  Method  recognizes  the  increased  importance  of  the  time-value 
of  financing  and  incorporates  fast-track  well. 

•  Method  enhances  the  effectiveness  and  incorporation  of  TQM, 
partnering,  team-building  and  fast-tracking  concepts. 

Coordination 

•  Single  entity  responsible  for  design  and  construction. 

•  Close  coordination  inherently  required  by  all  parties  leads  to 
quick  problem  resolution. 

•  Close  coordination  between  A/E  and  Contractor  occurs 
regarding  design  feasibility  and  constructability  issues. 

•  Design/Build  involves  Subcontractors  earlier  in  the  process 
obtaining  valuable  design  input. 

•  A/E  designs  to  contractor’s  strengths  facilitating  construction  . 

Table  2.1  Design/Build  Advantages  (Cont.)  (Roth  19951 


Area  of  Impact 

Design/Build  Advantages 

Coordination  (Cont.) 

•  The  new  organizational  make  up  within  Design/Build 

orgaiuzation  maximizes  the  respective  talents  and  experience 
of  all  the  project  players. 

Litigation 

•  Claims  and  litigation  are  limited  through  proper  risk  allocation 
and  assignment  of  responsibilities. 

•  Method  accommodates  multi-parameter  bidding  schemes 
which  allow  for  award  based  on  factors  other  than  price. 

•  Contractual  relationship  between  the  Owner  and  Design/Build 
entity  is  significantly  simplified 

•  Owner  is  insulated  from  liability  for  design  errors  and 
omissions.  Although  the  Design/Build  contractor  assumes 
responsibility,  he  is  empowered  with  the  ability  to  manage 
them  directly. 

Table  2,2  Design/Build  Disadvantages  (Roth  1995) 


Area  of  Impact 

Design/Build  Disadvantages 

Time 

•  Design/Build  contracts  may  take  longer  to  award  because  of 
the  complexity  of  the  award  process. 

•  Design/Build  process  is  more  dynamic,  requiring  increased 
team  and  administrative  participation. 

Cost 

•  Cost  of  responding  to  IFB  and  developing  proposal  can  be 
extremely  expensive.  This  tends  to  limit  competition  and 
eliminate  small  firms. 

•  Bonding  costs  for  A/E  and  Contractor  can  be  up  to  50% 
higher. 

•  Proposal  cost  is  a  sunk-cost,  recovered  only  if  contractor  is 
awarded  contract. 

•  Modifications  made  after  award  can  be  extremely  expensive 
if  not  made  in  a  timely  manner. 

•  Increased  responsibility  of  the  Design/Build  Contractor 
carries  increased  risk,  therefore,  he  may  increase  his  bid 
price  for  contingencies. 
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Table  2.2  Design/Build  Disadvantages  (Cont.)  (Roth  1995) 


Area  of  Impact 

Design/Build  Disadvantages 

Coordination 

•  A/E’s  direct  link  of  communication  with  owner  is  removed . 

•  A/E’s  first  allegiance  is  to  the  contractor  not  the  owner.  A/E’s 
feel  their  fiduciary  role  is  changed. 

•  Project  scope  must  be  defined  extremely  early  in  the  process. 

•  Process  can  be  a  real  risk  for  unsophisticated  owners  not 
familiar  Avith  their  administration . 

•  Knowledgeable  in-house  staff  must  closely  monitor  project. 

•  Importance  of  selecting  an  excellent  project  team  is  increased. 

•  Inexperienced  Subcontractors  dislike  the  imcertainty  of  the 
process. 

Legal 

•  Design/Build  contracts  are  prohibited  in  some  states. 

•  Litigation  may  develop  if  the  scope  of  work  defined  in  the  IFB 
is  not  absolutely  clear. 

Although  the  Design/Build  topic  is  very  interesting  and  seems  to  be  gaining 
momentum  as  a  contracting  vehicle,  the  intent  of  this  study  is  to  focus  on  the 
contracting  of  A/E  firms  that  ultimately  provide  the  drawings  and  specifications. 
The  purpose  of  quickly  reviewing  the  Design/Build  issue  is  to  show  that  there  is  a 
move  in  both  the  private  and  public  sector  to  stray  away  from  the  conventional 
Design/Bid/Build  is  occurring.  With  the  construction  industry  becoming  more 
competitive  and  international  companies  entering  the  U.S.  market,  owners  and 
construction  companies  in  the  private  sector  are  looking  for  better  ways  of  doing 
business,  or  areas  that  will  save  money. 

Apparently,  there  are  pressures  being  placed  on  A/E  firms  by  private 
owners  to  consider  price  as  part  of  the  selection  method.  This  is  contradictory  to 
the  Brooks  Act.  Has  the  private  industry  discovered  a  way  to  save  money  on 
design  costs  without  impacting  the  quality  of  construction?  There  is  probably  a 
yes  or  no  answer.  There  are  various  degrees  of  difficulty  of  construction  projects. 


The  bottom  line  is  there  has  never  been  a  valid  study  to  prove  or  disprove  the 
value  of  the  Brooks  Act. 

Thirty  years  ago  when  there  was  plenty  of  military  construction  dollars,  the 
main  focus  with  respect  to  A/E  firms  was  not  to  sacrifice  any  design  quality  for  the 
sake  of  saving  a  few  dollars.  Elaborate  design  aids  such  as  CAD  were  not  widely 
used.  Technology  with  computers  and  information  flow  are  now  making 
tremendous  advances  and  allowing  many  more  competitors  into  the  market.  With 
the  aid  of  technology,  many  simple  projects  can  be  reproduced  very  quickly  for  a 
fi'action  of  the  previous  costs.  Engineering  is  a  very  specialized  skill,  but  has 
technology  and  education  allowed  the  world  to  take  a  step  up  and  change  the  way 
projects  are  designed? 

Regardless  of  the  situation,  the  consensus  of  industry  participants  is  that 
good  design  is  essential  to  the  success  of  a  project.  Not  only  is  it  true  in  the 
United  States  but  countries  such  as  Japan  place  a  great  emphasis  in  the  design 
phase.  A  project  that  is  well  designed  will  experience  less  changes  and  ultimately 
give  the  owner  a  safer  and  more  maintenance  fi-ee  facility.  Since  85  percent  of  a 
facility’s  costs  are  experienced  after  the  building  is  completed  (Bell  et  al.  1990), 
why  are  some  private  owners  pressuring  A/E  firms  into  a  price-driven  qualification 
process? 

There  are  many  issues  and  questions  that  have  been  raised.  The  answers 
are  not  straight  forward,  but  before  attempting  to  even  address  the  issue,  a 
statistical  analysis  comparing  QBS  projects  to  non-QBS  projects  should  be 
conducted.  Is  the  quality  of  some  non-QBS  designed  facilities  the  same  as  QBS 
ones?  If  so,  then  the  construction  industry  can  probably  save  millions  of  dollars 


annually.  Obviously,  projects  that  are  complex  in  nature  and  are  on  the  cutting 
edge  of  technology  will  most  likely  require  the  best  A/E  firms.  If  there  is  a 
significant  difference  between  the  quality  of  non-QBS  projects  and  QBS  projects, 
then  the  Brooks  Act  needs  to  be  reinforced  as  the  best  way  to  do  business. 

Before  gathering  data  to  prove  or  disprove  the  hypothesis,  the  current 
laws,  regulations  and  policies  need  to  be  reviewed.  The  ultimate  goal  of  NAVFAC 
is  service  to  the  Naval  Fleet  through  innovative  ways  of  providing  better  facilities 
at  reduced  cost.  However,  the  avenue  chosen  to  give  the  best  service  has 
boundaries,  therefore,  it  is  important  to  examine  regulations  and  processes  that  are 
currently  being  used  in  NAVFAC. 

2.2  Laws,  Regulations  &  Policy 

There  are  basically  two  procurement  options  to  obtain  A/E  services  within 
NAVFAC  as  illustrated  in  Fugure  2.1.  Each  of  these  avenues  contain  eight  distinct 
steps  starting  with  the  original  procurement  strategy  to  the  ultimate  award  of  the 
contract.  Essentially,  the  A/E  services  can  be  obtained  by  separate  design  and 
construction  which  falls  into  the  traditional  method.  The  second  approach  is  to 
obtain  the  A/E  services  through  the  Design/Build  route.  For  the  purpose  of  this 
study,  separate  design  and  construction  will  be  the  focus.  In  Design/Build,  the 
design  fees  and  construction  costs  are  included  in  the  eight  step  process.  In  the 
traditional  sealed  bid  approach,  the  A/E  has  been  chosen  and  is  normally  included 
in  the  procurement  strategy  of  the  construction  contract.  (GSA  1994) 
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Figure  2.1  Procurement  Options 


Prior  to  examining  the  details  of  the  laws  and  regulations,  a  cursory  review 
of  the  history  of  the  major  events  leading  to  the  Brooks  Act  will  explain  the 
philosophy  or  the  intent  of  the  act  which  congress  passed  in  1972.  In  1947  the 
Armed  Services  Procurement  Act  gave  the  government  the  authority  to  contract. 
Naturally,  the  government  was  contracting  prior  to  these  dates  but  this  Act  gave 
official  direction.  Since  approximately  1939,  A/E  services  were  obtained  based  on 
qualifications  rather  than  price.  The  philosophy  is  quite  simple  and  clear.  When  a 
professional  registered  skill  is  required,  the  most  natural  approach  people  take  is  to 
attempt  to  get  the  most  qualified  person  within  reason.  Good  examples  are 
doctors  and  lawyers.  Architects  and  engineers  generally  fall  into  a  similar 
category,  especially  with  the  complex  design  of  buildings,  utilities  and  bridges. 
Proper  design  is  crucial  for  the  safety  to  the  public. 

Based  on  this  premise,  A/E  services  were  obtained  with  that  type  of 
philosophy.  Essentially,  the  government  was  conducting  business  in  this  fashion 
for  30  years.  Then  in  1967  the  Comptroller  General  recommended  that  price 
should  be  considered  in  the  selection  of  A/E  services  and  that  the  six  percent 
limitation  on  A/E  fees  should  be  repealed.  In  response  to  the  Comptroller’s 
recommendation,  H.  R.  12807  clarifies  procedures  for  the  selection  of  A/E  firms. 
In  summary,  the  bill  kept  the  current  practice  of  using  qualifications  for  selection, 
and  the  six  percent  limitation  fee  maintained.  (NAVFACCONTRACEN  1996) 

The  sbc  percent  fee  refers  to  the  preparation  of  designs,  plans,  drawings, 
and  specifications  and  can  be  no  more  than  six  percent  of  the  estimated 
construction  cost  of  the  project.  The  six  percent  fee  does  not  include  engineering 
services  such  as  field  investigations.  In  some  cases,  especially  in  the 
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environmental  arena,  the  cost  of  engineering  services  may  exceed  the  cost  of 
performance. 

In  1972,  the  Brooks  Act  was  passed.  The  law  spelled  out  specific 
procedures  for  A/E  contracting.  Congress  declared  its  policy  for  the  Federal 
Government  to  publicly  announce  all  requirements  for  architectural  and 
engineering  services,  and  to  negotiate  contracts  for  architectural  and  engineering 
services  on  the  basis  of  demonstrated  competence  and  qualification  for  the  type  of 
professional  services  required  and  at  fair  and  reasonable  prices.  (Brooks  Act  1972) 
The  law  is  very  logical  and  well  thought  out.  Since  engineering  is  a  specialized 
field,  it  is  prudent  to  hire  the  most  qualified  firm  as  long  as  the  prices  are  fair  and 
reasonable.  This  makes  even  more  sense  if  an  A/E  firm  is  selected  to  design  a 
complex  project  such  as  the  launching  pad  for  the  space  shuttle.  Not  only  is  that 
system  very  complex,  but  safety  is  another  concern  for  the  people  who  will  be  in 
the  shuttle.  That  situation  is  clear  cut,  i.e.  to  select  the  best. 

But  what  about  circumstances  such  as  the  design  of  a  perimeter  fence,  a 
softball  field,  or  an  office  building?  Does  it  make  sense  to  go  through  a  rigorous 
selection  process  to  choose  the  best  A/E  firm?  In  the  future,  will  owners  be  able 
to  purchase  off  the  shelf  designs  for  these  types  of  facilities  at  low  bid.  A  solid 
approach  to  analyzing  the  situation  is  to  link  the  method  of  selection  of  A/E  firms 
with  factors  that  relate  to  project  success.  This  is  a  very  objective  and  unbiased 
approach. 

Over  the  past  10  years.  Congress  has  been  moving  towards  improving  the 
contracting  process.  In  April  1985,  the  Competition  in  Contracting  Act  was 
passed  which  simply  amended  the  Armed  Services  Procurement  Act  by 


substituting  sealed  bids  for  formal  advertising  and  replaced  17  exceptions  with 
seven.  The  most  recent  law  was  passed  FAS  A  in  1995.  The  purpose  of  the  law 
was  to  speed  up  the  goverrunent’s  contracting  process  especially  with  small  A/E 
contracts.  Essentially,  this  raises  the  short  selection  process  (SSP)  threshold  from 
$25,000  to  $100,000. 

The  SSP  only  requires  interviews  over  the  phone  with  a  minimum  of  3 
firms.  There  is  no  longer  a  30  day  requirement  to  wait  for  a  response  from  A/E 
firms  to  receive  their  proposals.  The  Contracting  officer  can  designate  a 
reasonable  period.  Additionally,  the  A/E  firms  are  no  longer  required  to  turn  in  a 
Standard  Form  255.  This  form  describes  the  A/E’s  relative  experience  and 
provides  a  formal  proposal  tailored  to  a  specific  project.  The  only  form  required  is 
the  Standard  Form  254  which  is  the  qualifications  of  the  A/E  company  and  only 
needs  to  be  updated  on  an  annual  basis.  For  A/E  services  that  exceed  $100,000,  a 
Standard  Form  255  must  be  completed  on  each  contract. 

For  A/E  services  less  than  $100,000,  a  contract  can  be  awarded  in  one  of 
two  methods.  The  first  method  is  the  Chairman  of  the  pre-approved  standing 
board  can  select  an  A/E  firm  based  on  a  minimum  of  three  phone  interviews  and  a 
review  of  the  Standard  Form  254  submitted  or  ones  that  are  held  on  file.  Then  the 
Chairman  of  the  Board  gives  his  written  recommendation  to  the  Contracting 
Officer  for  signature.  The  other  method  is  a  standing  board  of  three  members 
appointed  by  the  Engineering  Field  Division  Commanding  Officer  who  can  directly 
approve  the  selection  of  an  A/E  firm.  However,  the  board  must  then  write  a  report 
on  why  a  the  A/E  firm  was  selected. 
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Every  field  office  varies  with  respect  to  organizational  structure  and 
mission.  Obviously,  to  set  one  specific  guideline  to  meet  all  the  objectives  would 
essentially  be  impossible.  Therefore,  the  Department  of  Defense  (DoD)  falls  into 
the  broad  category  of  the  Federal  Acquisition  Regulations  (FAR)  and  the  DoD 
FAR  supplement.  These  are  the  primary  procurement  regulations  for  the  DoD. 
The  Navy  Acquisition  Procedures  Supplement  (NAPS)  provides  additional 
guidance  in  the  area  of  Navy  contracting.  NAVFAC  has  further  defined  the 
guidance  to  include  specific  authority,  responsibilities  and  internal  procedures. 
The  NAVFAC  Publication  P-68  is  the  source  of  information  for  the  delegation  of 
authorities  for  the  execution  and  administration  of  A/E  contracts.  Finally,  the 
Engineering  Field  Divisions  (EFD)  set  more  refined  guidance  tailored  to  the 
specific  organizational  needs  and  goals.  (NAVFACCONTRACEN  1996) 

2.3  Typical  NAVFAC  Acquisition  Process 

FAS  A  has  enabled  the  government  to  expedite  many  contract  actions.  The 
trend  is  to  provide  the  services  to  the  customers  without  jeopardizing  competition. 
Depending  on  the  situation  and  the  amount  of  A/E  contracting,  the  tjq)ical 
acquisition  process  for  A/E  contracts  greater  than  $100,000  is  shown  in  Table  2.3. 
(NAVFACCONTRACEN  1996) 


Table  2.3  NAVFAC  A/E  Acquisition  Process.  (NAVFACCONTl 

ElACEN  1996) 

Process 

Action 

Time 

Requirement  identification 

User 

Varies 

Acquisition  planning 

EFDPM 

Advertisement 

Contracts 

2  Months 

Qualification  statements  received 

Evaluation  Board 

1  Month 

Slate/preselection  board  meeting 

Evaluation  Board 

2  Weeks 

Slate/preselsction  board  report 

Chairman  of  the  Board 

1  Week 

Request  for  Proposal 

Contracts 

1  Week 

Government  estimate  preparation 

EFDPM 

2  Weeks 

Receipt/review  of  proposal 

A/E  Firm 

2  Weeks 

DCAA  Audit  (over  $500K) 

Contracts 

■esesbi 

Prenegotiation  objectives  established 

Contracts/Tech  Rep 

2  Weeks 

Negotiation 

Contracts/Tech  Rep 

Post  Negotiation  Memorandum 

Contracts 

1  Week 

Contract  Award 

Contracts 

1  Week 

Contract  Administration 

Contracts/ROICC 

Varies 

2.4  Length  of  Acquisition  Process 

In  1989,  NAVFAC  conducted  a  study  to  determine  how  long  it  took  to 
obtain  A/E  services.  The  results  showed  that  from  the  time  a  project  was 
authorized  until  the  contract  was  signed,  330  days  had  elapsed.  This  was 
unacceptable  to  NAVFAC.  Private  industry  would  go  out  of  business  waiting 
nearly  a  year  to  select  their  design  firms.  As  a  result  of  the  study  conducted  in 
1989,  NAVFAC  made  some  recommendations  which  led  to  (NAVFACENGCOM 


•  Less  time  required  to  award  an  A/E  contract. 

•  More  responsive  to  customer  needs. 

•  Better  customer  satisfaction. 

•  Increased  quality  of  A/E  selection  and  supporting  document 

•  Reduced  paperwork. 

•  Enhanced  teamwork  and  ownership  of  the  outcome. 

•  More  efficient  manpower  utilization 

As  part  of  this  study,  a  brief  survey  was  conducted  by  telephone 
interviewing  Engineering  Field  Divisions  and  Engineering  Field  Activities, 
(EFD/EFA)  within  NAVFAC.  The  interviews  conducted  include  the  following 
four  Commands;  EFA  Northwest,  EFA  Chesapeake,  EFD  Southwest  and  EFD 
Pacific.  The  result  of  survey  indicates  that,  the  average  procurement  time  has  been 
reduced  to  four  months,  a  seven  month  improvement  in  the  acquisition  process. 
This  supports  information  provided  in  the  curriculum  taught  at  the  Naval  Facilities 
Engineering  Command  Contracts  Training  Center  which  states  that  the  normal 
procedure  for  contracting  an  A/E  firm  should  take  117  days  for  a  standard  project. 
Table  2.4  shows  the  time  fi'ames  associated  with  different  project  priorities. 

The  time  fi-ames  for  the  four  other  methods  vary  fi’om  32  to  74  days  to 
award  a  contract.  Track  B,  Fast  Track,  essentially,  goes  through  the  same  process 
as  the  standard,  but  involves  more  senior  management  interest  and  the  contract  is 
hand  carried  through  the  process.  Track  C,  Generic  Slate,  are  routine  projects  that 
are  less  than  $10  million.  Basically,  a  slate  of  three  firms  are  chosen  from  a  pre¬ 
advanced  list  by  region.  The  updating  of  the  list  is  done  twice  a  year.  A  standing 
slate  board  then  chooses  from  3  interested  firms.  Essentially,  the  lengthy  process 
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of  formal  slate/select  boards  is  eliminated.  Track  D,  Urgent  and  Compelling,  is 
used  if  the  project  is  considered  mission  essential,  an  operational  necessity,  or  the 
customer  certifies  urgency.  In  any  case,  a  Justification  and  Authorization  (J  &  A) 
must  be  signed  for  this  approach.  Finally,  Track  E,  the  Indefinite  Quantity,  (IDQ) 
contracts  are  used  as  needed  by  type  of  contract.  Normally  a  minimum  and 
maximum  amount  are  stated  on  the  contract.  One  large  contract  is  awarded  to  an 
A/E  firm  and  as  design  services  are  needed  within  scope  of  the  originally  contract, 
a  delivery  order  is  written  that  specifies  the  requirements  and  specific  scope  of 
work  to  be  accomplished.  (NAVFACCONTRACEN  1996) 


Table  2.4  Priority  System.  (NAVFACCONTRACEN  1996) 


Track 

HHIHiSSBSSHiHHii 

Time 

A 

Standard  Process 

117 

B 

Fast  Track 

74 

C 

Ready/Generic  Slate 

37 

D 

Urgent  and  Compelling 

32 

E 

Indefinite  Quantity 

51 

While  the  study  conducted  in  1989  led  to  the  initiative  as  mentioned 
previously,  NAVFAC  also  gave  the  EFD/EFA/PWC’s  flexibility  to  adapt  work 
processes  to  meet  the  user’s  needs  while  maintaining  their  own  organizational 
goals. 

Even  more  important  today  with  the  closing  of  bases  and  restructuring,  the 
Navy  continues  to  look  at  ways  to  reduce  manpower  while  improving  capability. 
Technology  will  play  the  primary  role  in  being  able  to  accomplish  this  task.  That  is 
why  it  is  important  to  periodically  address  questions  such  as  does  it  make  sense  to 
spend  valuable  resources  and  money  trying  to  choose  the  most  qualified  A/E  firm 
for  every  project?  FASA  has  resolved  the  time  length  issues  for  small  contracts. 


Why  not  review  current  projects  and  base  decisions  on  the  successes  and  failures 
of  projects  rather  than  a  set  policy  based  on  anecdotal  information?  The  A/E 
selection  procedure  really  has  not  been  reviewed  since  the  early  1970’s.  The  same 
philosophy  holds  true  for  Design/Build  projects.  Now  that  the  Navy  has  been 
using  design  build  for  over  five  years,  it  is  time  to  conduct  statistical  study  to 
analyze  project  data  to  determine  how  much  benefit  or  impact  the  Design/Build 
process  has  versus  the  Design/Bid/Build. 

2.5  Phase  I  Background 

The  following  discussion  fi-om  the  Phase  I  Background  through  the  next 
two  Chapters  is  taken  from  the  Phase  I  report  A/E  Service  Selection  Method 
which  was  generated  by  the  Department  of  Civil  Engineering  at  the  University  of 
Texas  at  Austin  (Vanden  Bosch  et  al.  1996). 

Key  participants  within  the  construction  industry  report  a  growing  new 
trend  in  design  firm  selection  fi-om  selection  based  solely  on  the  qualifications  of 
the  design  firm  towards  selection  based  solely  on  price.  However,  selection  based 
solely  on  price  alarms  many  design  firms  and  professional  organizations,  who  feel 
that  such  a  focus  will  substantially  degrade  the  quality  of  the  design  service.  This 
will  in  turn  result  in  completed  projects  that  do  not  fully  meet  client  needs  and 
expectations. 

Reoccurring  attempts  have  been  made  both  to  weaken  and  to  strengthen 
the  Brook’s  Act.  To  date,  the  act  has  survived  intact,  but  is  under  increasing 
attack.  The  Justice  Department  has  required  professional  societies  to  remove 
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ethics  clauses  in  their  by-laws  that  prevented  the  competitive  bidding  of  services  on 
the  grounds  that  such  clauses  violated  anti-trust  laws.  Thus  inherent  tensions 
between  pricing  pressure  and  qualifications  pressure  for  design  services  are  likely 
to  continue.  The  Brook’s  Act  is  not  without  its  critics,  who  charge  that  the  act  is 
inherently  biased,  discriminatory,  and  restricts  competition.  These  criticisms  are 
further  explored  in  Chapter  3. 

The  current  arguments  both  for  and  against  QBS  are  based  on  anecdotal 
data.  The  problem  with  anecdotal  examples  is  that  there  is  no  test  to  determine  if 
they  are  extreme  examples  of  infrequently  occurring  events,  or  to  what  degree  the 
examples  represent  the  central  tendencies  of  the  general  population.  Because  of 
this,  anecdotal  examples  cannot  be  used  to  establish  population  trends  or  even  to 
prove  a  point.  Therefore,  a  statistical  study  needs  to  be  performed  to  evaluate  the 
efficacy  of  QBS. 

2.6  Objectives  of  Phase  1 

Phase  I  set  up  the  fi'amework  for  this  study.  The  primary  objectives  of 
Phase  I  were  to: 

1)  Develop  data  gathering  procedures  and  tools  for  further  study, 

2)  Address  sample  size  and  availability  of  data  for  further  study, 

3)  Conduct  reviews  of  the  proposed  methodology  in  order  to  refine 
gathering  procedures  and  tools, 

4)  Make  necessary  refinements  in  the  procedures  and  tools,  and 

5)  Communicate  all  findings  in  a  report. 

2.7  Scope  of  Phase  I 

The  scope  of  Phase  I  was  to  develop  statistically  sound  survey  tools  for 
gathering  relevant  project  information  linking  a  project’s  performance  and  the 
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method  of  selection  of  its  design  service  provider.  The  scope  of  research  is 
therefore  to: 

1.  Build  on  prior  studies 

In  any  study  involving  substantial  issues  and  possibly  affecting  policy 
and/or  legislation,  it  is  incumbent  on  the  research  team  to  extensively  search 
existing  literature  for  similar  or  related  studies,  and,  if  such  studies  exist,  build  on 
their  results.  Phase  I  of  this  study  includes  such  an  effort.  The  results  of  the 
literature  review  are  presented  in  Chapter  3 . 

2.  Develop/Modify  tools 

Every  statistical  study  is  unique.  The  researchers,  therefore,  must  develop 
a  survey  instrument  and  approach  that  meets  the  needs  of  this  study.  The  survey 
instrument  from  phase  I  has  been  refined  and  is  given  in  appendix  A. 

2.  Address  statistical  issues 

There  are  many  issues  associated  with  statistical  analysis  that  can  affect  the 
usability  of  data..  Failure  to  account  for  these  issues  could  damage  the  analysis 
effort.  These  issues  are  briefly  discussed  at  the  end  of  Chapter  3  and  further 
elaborated  in  Chapter  4. 

4.  Get  preliminary  industry  reaction  to  methodology  and  tools 

The  proposed  exploratory  survey  methodology  and  data  gathering 
approach  were  evaluated  by  independent  individuals  knowledgeable  in  the  A/E/C 
industry.  Improvements  were  made  based  on  their  suggestions.  Preliminary 
feedback  on  the  draft  methodology  and  survey  tools  is  presented  in  Chapter  3. 


2.8  Methodology  of  Phase  I 

The  methodology  employed  in  Phase  I  is  depicted  in  Figure  2.2  and  has 
been  completed.  This  study  is  the  pilot  test  of  the  survey  tools  developed  in 
Phase  I. 


Figure  2.2  Methodology  of  Phase  I 


Chapter  3:  Preliminary  Investigation 

3.1  Literature  Summary 

An  extensive  literature  search  was  performed  during  Phase  I,  and  an 
annotated  bibliography  of  the  relevant  literature  is  presented  in  the  Phase  I  report. 
Findings  of  the  literature  review  are  summarized  below.  (Vanden  Bosch  et  al. 
1996) 


The  Brook’s  Act 

As  previously  discussed,  the  Brook’s  Act  controls  how  federal  agencies 
select  design  firms  for  federal  projects.  The  act  requires  federal  agencies  to  invite 
at  least  three  architect  or  engineering  firms  to  be  considered  for  each  selection 
process.  These  firms  are  then  ranked  in  order  of  their  qualifications  alone.  The 
highest  ranked  firm  is  selected.  Pricing  considerations  are  addressed  in  the  Act  by 
requiring  the  agency  to  negotiate  only  with  the  highest  ranked  firm,  and  if 
negotiations  are  successful,  award  to  that  firm.  If  the  negotiations  are 
unsuccessful,  the  agency  terminates  that  selection  and  moves  on  to  the  next  highest 
qualified  firm.  (FAR  36.6) 

Critics  of  the  Brook’s  Act  complain  that  the  act  is  inherently  biased  toward 
large,  established  firms,  and  discriminates  against  new,  small,  and  minority  firms. 
(Hampton  1994)  Such  criticisms  have  appeared  in  court  cases  contesting  the 
selection  of  one  design  firm  over  another.  Without  disputing  such  allegations,  the 
federal  courts  have  nevertheless  upheld  the  Brook’s  Act,  and  it  has  withstood  all 
such  challenges.  (485  Federal  Supplement  1292;  B-201395.2  1982;  B-218404.2; 


B-2 18474)  Another  concern  is  that  even  though  the  Federal  Acquisition 
Regulations  (FAR)  declares  that  the  selection  process  promulgated  by  the  Brook’s 
Act  must  meet  the  requirements  of  “fair  and  open  competition”  by  law,  in  fact  it  is 
restricted  competition  with  sole-source  negotiations,  two  terms  which  experienced 
public  procurement  officials  usually  associate  with  exorbitantly  higher  prices. 
(FAR  36.6) 

To  counter  against  the  assaults  on  the  Brook’s  Act,  the  major  professional 
societies  have  formed  an  organization  to  review  and  study  the  issue  of  how  best  to 
defend  the  Brook’s  act.  That  organization  is  the  Committee  on  Federal 
Procurement  of  Architecture  and  Engineering  Services  (COFPAES).  COFPAES’ 
interest,  though  primarily  federal,  extends  to  the  state  and  local  levels.  In  addition 
to  COFPAES,  a  recent  report  from  Massachusetts  presents  anecdotal  information 
supporting  the  Brook’s  Act’s  use.  (AIA  1996;  Massachusetts  Taxpayer’s 
Foundation  1995) 

There  are  many  processes  in  use  for  selecting  the  design  service  provider. 
Perhaps  the  most  common  classification  is  a  modification  of  the  classification 
system  used  by  the  Professional  Services  Management  Journal  (PSMJ)  in  their 
annual  design  fee  survey.  (PSMJ  1996)  The  modification  of  the  PSMJ 
classification  rather  neatly  proceeds  from  pure  price-based  selection,  to  pure 
qualifications  based  selection  and  was  given  earlier  in  Table  1.1. 

Price-Based  Selection  Trends 

The  trend  towards  price-dominated  selection  of  A/E  services  appears  to  be 
leveling  off.  There  is  some  indication  that  the  practice  is  abating  as  owners  and 


others  learn  that  the  anticipated  costs  savings  do  not  occur  in  the  long  run. 
(Architectural  Record  1994;  Hathway  1995;  Ichniowski  etal.  1995;  PSMJ  1996) 
Price-only  selection  comes  from  private  owners,  other  service  providers, 
construction  contractors,  and  public  agencies  at  the  federal,  state  and  local  levels. 
Pricing  considerations  in  the  selection  of  a  design  service  provider  are  significant  in 
the  public  sector,  as  well  as  the  private  sector.  By  law,  some  states  require  the 
design  service  provider  to  bid  for  public  projects.  Many  other  government  and 
quasi-government  agencies,  districts,  boards,  and  other  public  service  providers 
appear  to  practice  selection  of  the  project  designer  based  only  on  price,  regardless 
of  laws  apparently  to  the  contrary.  Surprisingly,  A/E  firms  subcontracting  work  to 
smaller  consulting  firms  often  do  so  based  primarily  on  price.  (Angelo  1995; 
Schriener  1995;  Massachusetts  Taxpayer’s  Foundation  1995;  PSMJ  1996) 

Public  sector  price  pressures  driving  selection  of  design  service  may  be 
attributed  in  part  to  the  Federal  Acquisition  Regulations  (FAR)  Part  6  - 
Competition  Requirements,  and  in  particular  to  Suhpart  6.1  -  Full  and  Open 
Competition,  where  paragraph  6.101  Policy  establishes  full  and  open  competition 
as  the  normal  policy  of  the  land.  All  other  procedures  are  limited  exceptions 
usually  requiring  a  written  justification  and  higher  authority  approval,  unless  a 
statutory  exception  (such  as  the  Brook’s  Act)  provides  otherwise.  Moreover, 
Suhpart  6.5  —  Competition  Advocate,  requires  each  agency  to  appoint  a 
competition  advocate  to  actively  promote  full  and  open  competition.  Finally,  the 
Brook’s  Act  itself  appears  to  be  mandatory  only  when  the  contracted  services 
require  a  licensed  professional  A/E  to  perform  them.  Those  A/E  services  not 
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requiring  a  professional  license  and  signature  may  not  be  mandatory.  (Lunch  1994; 
USCA  40;  B-201395.2  1982;  FAR  Part  6) 

The  public  policies  embodied  in  the  FAR  regarding  competition  are  rooted 
in  a  pronounced  desire  by  taxpayers  to  minimize  their  tax  burden.  Taxes  are  the 
pre-dominate  source  of  government  revenues  for  public  projects.  One  of  the 
consequences  of  this  is  reflected  in  public  policies  at  all  levels  of  government  to 
obtain  goods  and  services  for  public  use  at  the  lowest  reasonable  price.  The 
predominate  method  adopted  by  nearly  all  public  procurement  offices  to  minimize 
the  cost  of  such  goods  and  services  is  to  define  the  requirements,  and  publicly 
invite  all  interested  parties  to  submit  their  price  (bids)  for  providing  the  goods  or 
services.  (Lunch  1991;  Korman  etal  1995;  ENR  1991;  DeFraites  1989) 

Private  sector  price  pressures  driving  selection  of  design  services  may  be 
attributed  to  the  perception  that  design  services  are  a  commodity,  caused  in  part  by 
design  firms  limiting  their  liability  to  the  owner.  The  extensive  use  of  standard 
form  design  contracts  also  adds  to  this  view.  The  perception  that  there  is  little 
difference  between  A/E  firms  is  pervasive.  (Architectual  Record  1994;  Korman 
etal.  1995;  Grogan  1995;  PSMJ  1996) 

Project  Performcmce 

In  a  series  of  meetings  held  in  Fall  1995,  the  research  team  developed  a  list 
of  project  outcome  and  project  process  performance  criteria  based  on  team 
expertise,  and  significant  prior  research  work  performed  by  the  Construction 
Industry  Institute.  (Cn  et  al.  1986-1994)  The  project  performance  categories 
include: 


•  Total  installed  cost  (TIC)  cost  performance 

•  TIC  cost  performance  relative  to  industry  benchmark 

•  Unit  construction  cost 

•  Cost  of  design  quality  relative  to  TIC 

•  Cost  of  construction  quality  relative  to  TIC 

•  $  amount  of  claims  or  litigation  relative  to  TIC 

•  Overall  schedule  performance  relative  to  owner  goals 

•  Overall  schedule  performance  relative  to  industry  benchmark 

•  Jobsite  safety  performance  relative  to  goal 

•  Jobsite  safety  performance  relative  to  industry  benchmark 

•  Level  of  customer  (owner)  satisfaction 

•  Plant/facility  output  performance  relative  to  owner  requirements 

•  Safety  of  facility  operations  and  maintenance 

•  Aesthetics  of  facility 

•  Cost  efficiency  of  facility  operations 

•  Cost  efficiency  of  facility  maintenance 

•  Job  profit  of  A/E 

•  Job  profit  of  contractor 

•  Satisfied  project  personnel 

•  Accessibility  to  new  service  providers 

•  Ease  of  overall  project  delivery  from  owner's  perspective 

•  Amount  of  required  owner  front-end  costs 

•  Fair  system  of  selection  with  integrity  and  minimal  corruption, 
abuse,  or  protests 

•  Ease  and  uniformity  in  execution  of  selection  process 

These  project  outcome  and  project  process  performance  categories  were 
used  to  develop  the  survey  instruments  as  described  in  Chapter  4.  Issues  of 
diversity,  complexity,  and  difficulty  in  measurement  were  addressed  by  the 
research  team  as  well. 

There  are  many  influences  to  project  success.  One  of  the  most  important 
influences  is  in  the  design  stage  of  the  project,  where  small  changes  in  the  design 
process  may  have  enormous  cost  consequences  in  construction  or  operations. 
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Statistical  Studies 

The  literature  review  revealed  no  rigorous  statistical  studies  concerning 
qualifications-based  selection  of  A/E  services.  The  study  will  therefore  be 
exploratory  in  nature  and  should  include  multiple  sources  of  data  and  data 
gathering  methods  as  discussed  in  Chapter  4. 

3.2  Overview  of  Statistical  Procedures 

Several  methods  exist  for  statistical  analysis  of  data.  All  generally 
acceptable  methods  today  are  founded  in  probability  theory  and  a  rigorous  and  a 
carefully  executed  data  collection  process.  (Knoke  etal.  1982;  Babbie  1990)  To 
roughly  frame  the  available  analysis  techniques,  two  broad  techniques  are  used 
below:  means  and  variance  testing,  and  analysis  of  variance.  The  Phase  I  report 
addressed  these  techniques  in  detail  (Vanden  Bosch  et  al.  1996) 

Of  the  many  thousands  of  projects  that  could  be  examined,  there  is  time 
and  money  for  only  a  limited  sampling  of  the  projects  that  exist.  Generally, 
completed  projects  are  necessary  to  measure  overall  project  performance,  so  all 
uncompleted  projects  will  be  excluded.  Much  of  the  data  required  for  this  study  is 
not  likely  to  be  kept  in  a  single  file,  or  even  in  a  single  location.  In  addition,  much 
of  the  data  is  probably  resident  only  in  the  minds  of  the  key  project  participants. 
Therefore,  the  availability  of  data,  and  the  availability  of  knowledgeable  persons,  is 
a  requirement.  Projects  completed  more  than  five  years  ago  will  be  excluded  on 
the  basis  that  the  time  lag  for  personal  recall  is  too  great  to  be  reliable. 


Projects  too  small  or  too  large  in  size  may  disproportionately  distort  the 
analysis  results.  Therefore  projects  less  than  $2  million  and  more  than  $60  million 
are  expected  to  be  excluded  from  the  study. 

The  diversity  of  projects  in  the  universe  of  all  construction  projects  is  too 
great  to  include  all  types  of  projects.  It  was  proposed  to  limit  the  project  sectors 
to  buildings,  horizontal  construction,  and  light  industrial. 

The  building  sector  consists  of  general  buildings  and  low-rise  structures 
such  as  jails,  prisons,  correctional  facilities,  schools,  maintenance  facilities,  office 
buildings,  retail  stores,  but  not  high-rise  buildings. 

The  horizontal  construction  sector  consists  of  roads  and  highways,  earth 
dams  and  levees,  flood  control  structures,  irrigation,  power  transmission  systems, 
water  distribution  systems,  sewage  collection  systems,  and  grading,  excavation  or 
landscaping. 

The  light  industrial  sector  consists  of  power  plants,  water  treatment  plants, 
and  sewage  treatment  plants. 

3.3  Preliminary  Interview  Feedback 

The  developed  survey  tools  and  proposed  methodology  were  reviewed 
with  knowledgeable  industry  officials  to  provide  a  reality  check  on  their  direction, 
applicability,  and  completeness.  In  addition,  the  interviews  uncovered  areas  that 
require  further  attention.  These  recommendations  were  incorporated  into  the 
survey  design  and  methodology.  (Vanden  Bosch  1996) 


3.4  Pretest 


This  thesis  addresses  the  need  to  further  test  the  survey  questionnaires  on  a 
small,  select  sample  in  order  to  get  detailed  feedback.  The  next  chapters  give 
detail  of  this  effort  along  with  initial  results  of  this  small  sample.  In  addition,  a 
brief  critique  of  the  Phase  I  survey  instruments  is  given  as  well. 


Chapter  4:  Methodology 


4.1  Hypothesis 

In  statistical  analysis,  a  hypothesis  is  a  statement  of  the  intent  of  the 
statistical  analysis,  i.e.,  what  it  is  the  study  is  attempting  to  establish.  For  this 
study  the  hypothesis  is,  in  its  simplest  form,  that  project  performance  is  related  to 
the  method  of  selection  of  the  designer.  More  specifically,  project  performance  is 
negatively  related  to  pricing  pressures  exerted  on  the  design  service  provider 
during  his/her  selection  process.  A  hypothesis  cannot  be  directly  proven,  so  its 
opposite,  or  null  position  is  disproved.  (Knoke  etal.  1982;  Babbie  1990)  The  null 
hypothesis  for  this  study  is  that  there  is  no  significant  relation  between  project 
performance  and  the  method  of  selection  of  the  design  service  provider. 


4.2  Methodology 


This  section  presents  the  steps  and  data  collection  techniques  that  were 
executed  and  discusses  statistical  analysis  issues  affecting  the  usability  of  the  study 
results.  Figure  4.1  shows  the  methodology  for  the  proposed  study.  The  work 
accomplished  in  this  thesis  has  addressed  the  methodology  on  a  small  scale  in 
order  to  pilot  test  the  survey  approach. 


Figure  4. 1  Methodology 
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4.3  Data  Gathering  Procedures 

The  sampling  frame  was  developed  from  available  sources.  Once  the 
sample  projects  were  identified,  key  individuals  for  each  project  in  the  sample  were 
contacted.  Personal  contact,  either  by  telephone  or  face-to-face,  was  made  with 
each  key  project  individual. 

Once  the  questionnaires  were  delivered,  follow-up  interviews  were  made  as 
required  to  address  questions,  or  to  maintain  and  induce  enthusiasm  in  the 
respondents  to  complete  the  questionnaires,  thus  insuring  a  reasonable  sample 
response  rate.  Data  quality  was  monitored  as  the  data  were  received,  and  follow¬ 
up  contacts  made  to  resolve  unclear  and  missing  data. 

4.4  Data  Analysis  Procedures 

Meaningful  data  analysis  is  largely  dependent  on  receiving  all  or  most  of 
the  sample  data.  While  the  major  focus  of  this  study  was  project  performance  and 
the  A/E  selection  method,  there  were  other  factors  that  affected  project 
performance.  Not  every  factor  below  affected  this  pilot  study,  but  are  included 
on  the  list  to  show  the  factors  that  were  taken  into  account: 

(1)  Externally  driven  factors,  such  as 

•  competitive  market  conditions 

•  local  regulatory  and  permit  requirements 

•  labor  strikes 

•  material  availability 

•  abnormal  weather 

•  civil  unrest,  war,  and  insurrection 

•  acts  of  God 
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(2)  Owner  controlled  events,  such  as 

•  pre-project  planning 

•  contracting  strategy 

•  design  reviews 

•  constructability  program 

•  construction  manager 

•  time  constraints 

•  use  of  incentives 

•  financing  or  funding  constraints 

•  partnering 


(3)  Project  specific  factors,  such  as 

•  Remoteness  of  the  site 

•  complexity  of  the  project 

•  uniqueness  of  the  project,  e.g. ; 

•  prototype 

•  first  of  a  series 

•  new  process  or  technique 


(4)  Constructor  controlled  events,  such  as 

•  experience 

•  proclivity  to  claim 

•  attitude 

•  safety  practices 

Project  success  is  a  combination  of  cost,  schedule,  quality  and  owner 
satisfaction.  All  of  the  above  factors  can  independently  affect  any  and  all  of  these 
project  success  measures.  Therefore  it  is  necessary  to  include,  or  account  for,  the 
influence  of  these  factors  in  the  data.  (Knoke  etal.  1982;  Babbie  1990)  Once  all 
these  other  influencing  factors  were  controlled  for  in  data  collection,  the  resulting 
data  was  analyzed  regarding  project  performance  and  A/E  selection  method. 


At  this  point  in  the  analysis,  the  sub-sample  groupings  were  tested  to  see  if 
they  were  significant,  i.e.,  if  they  were  statistically  different  from  each  other.  If  the 
sub-sample  groupings  were  significant,  then  two  questions  remain: 

1)  “Do  the  project  success  factors  co-vary  with  the  A/E  selection 
methods?”,  and  if  so, 

2)  “How  strong  is  the  co-variance  relationship?” 
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4.5  Data  Gathering  Tools 

Based  on  the  project  performance  categories  presented  in  Chapter  3,  a 
matrix  of  questions  and  requirements  was  developed  as  shown  in  Table  4.1 
corresponding  to  the  questionnaire  presented  in  Appendix  A. 


TABLE  4.1  Performance  Criteria/Questionnaire  Matrix 


Performance  Criteria 

m 

Survey  Question(s) 

(D 

OUTPUTS: 

1.  TIC  Cost  Performance 

2.2.1 

2.  TIC  cost  performance  relative  to  industry 
benchmark 

2.2.1  and  2.1.6  (make  our  own 
sample  comparison) 

3.  Unit  construction  cost 

2.1.6 

4.  Cost  of  design  quality  relative  to  TIC 

2.2.1  and  changes  questionfpossibly) 

5.  Cost  of  construction  quality  relative  to  TIC 

2.2.1  and  changes  question 
(possibly)  (4.  and  5.  are  difficult  and 
probably  understated) 

6.  $  amount  of  claims  or  litigation  relative  to  TIC 

2.6.1  and  2.2.1 

7.  Overall  schedule  performance  relative  to  owner 
goals 

2.3. 1-6 

8.  Overall  schedule  performance  relative  to  industry 
benchmark 

2.1.6  and  2.3. 1-6  (make  our  own 
sample  comparison) 

9.  Jobsite  safety  performance  relative  to  goal 

10.  Jobsite  safety  performance  relative  to  industry 
benchmark 

N.A. 

11.  Level  of  customer  (owner)  satisfaction 

2.7.1.a;  2.7.8 

12.  Plant/facility  output  performance  relative  to 
owner  requirements 

2.5.1;  2.5.2 

13.  Safety  of  facility  0  &  M 

2.7.6.a-c 

14.  Aesthetics  of  facility 

2.1 3. c 

15.  Cost  efficiency  of  facility  operations 

2.7.5a-c 

16.  Cost  eflSciency  of  facility  maintenance 

2.7.6.a-c 

17.  Job  profit  of  A/E 

N.A. 

18.  Job  profit  of  contractor 

N.A. 

19.  Satisfied  proiect  personnel 

N.A. 

20.  Accessibility  to  new  service  providers 

N.A, 

21.  Ease  of  overall  project  delivery  flrom  owner's 
perspective 

N.A, 

22.  Amoimt  of  required  owner  front-end  costs 

N.A. 

23.  Fair  system  of  selection  with  integrity  and 
minimal  corruption,  abuse,  or  protests 

2.6.1;  2.6.2 

24.  Ease  &  uniformity  in  execution  of  selection 
process 

N.A. 
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Using  this  matrix,  the  project  performance  questionnaires  were  developed. 
Three  variations  of  the  project  performance  questionnaire  were  developed, 
however,  only  the  one  for  the  building  sector  was  used  in  this  pilot  study. 

Column  2  of  Table  4.1  shows  the  question  number  in  the  project 
performance  questionnaire.  Issues  in  Table  4.1  without  corresponding  questions 
are  shown  by  “N.A.”  in  column  2  of  the  table.  Such  issues  are  not  addressed 
because  the  research  team  felt  they  were  too  hard  to  obtain  data,  or  were  too 
sensitive. 

In  addition  to  the  performance  measurement  questions  generated  from 
Table  4.1,  questions  to  control  for  the  other  factors  that  affect  project  success, 
discussed  in  the  antecedent  Data  Analysis  Procedures  section,  are  included  in  the 
project  performance  questionnaire  under  question  2.1.7.  These  questions  are 
designed  to  allow  the  isolation  of  the  effects  of  the  A/E  selection  method  on 
project  success  factors  in  the  data  analysis. 

Finally,  questions  are  included  in  the  questionnaires  to  probe  respondents  in 
terms  of  project  type,  designer,  background  information  on  the  project, 
identification  of  key  personnel,  participating  firms  in  the  project,  etc.  These  types 
of  questions  are  at  the  beginning  of  the  questionnaires.  See  appendix  A  for  the 
final  copy  of  the  project  performance  questionnaire. 
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4.6  Service  provider  questionnaire 

The  A/E  point  of  contact  (POC)  was  included  in  the  project  performance 
questionnaire.  Once  this  information  was  received,  a  separate  service  provider 
questionnaire  was  sent  to  the  A/E. 

The  service  provider  questionnaire  was  developed  in  brainstorming 
sessions  by  the  research  team  incorporating  the  results  of  the  literature  review.  A 
separate  questionnaire  was  developed  because  it  became  clear  that  the  design 
service  provider  was  best  positioned  to  answer  those  questions,  while  the  owner 
was  best  positioned  to  answer  the  project  performance  questionnaire.  Projects 
were  segregated  into  QBS,  non-QBS,  or  a  combination  of  the  two  based  on 
responses  to  this  questionnaire.  See  appendix  B  for  the  final  copy  of  the  service 
provider’s  questionnaire. 


Chapter  5:  Presentation  and  Data  Analysis 
5.1  Characterization  of  Sample 

The  pilot  sample  comes  from  a  combination  of  two  sources.  The  first 
source  was  generated  by  F.  W.  Dodge  who  provided  461  randomly  selected 
building  sector  projects.  These  projects  have  been  completed  within  the  last  year 
and  are  representative  of  America’s  construction  industry.  The  other  source  of 
projects  came  from  locally  selected  projects  in  the  Austin,  Texas  area  based  on 
personal  contacts.  From  the  461  projects,  nine  projects  were  randomly  selected 
within  the  state  of  Texas  and  three  projects  were  chosen  locally.  The  scope  of  the 
sample  size  was  limited  to  Texas  because  of  funding  and  timing  constraints. 

This  study  will  set  the  foundation  for  the  next  research  phase  which  will  be 
conducted  on  a  much  broader  scale.  The  more  comprehensive  analysis  will 
include  projects  encompassing  the  United  States  and  will  also  include  the 
horizontal  and  utilities  sectors  of  construction.  As  discussed  earlier,  this  study  is  a 
pilot  study  to  determine  the  response  rate  and  quality  of  questionnaire.  Based  on 
the  feedback  provided  by  the  initial  sample,  the  questionnaire  will  be  modified. 

The  specific  types  of  projects  chosen  are  schools,  apartment,  and  office 
/retail  buildings.  The  size  of  projects  range  from  70,000  to  320,000  gross  square 
feet  (GSF)  and  total  installed  cost  of  4  to  25  million.  The  response  rate  of  all  the 
questionnaires  sent  out  was  66  percent.  As  previously  mentioned  in  the 
methodology,  the  owners  were  initially  contacted  by  phone  and  ask  if  they  were 
willing  to  participate.  Once  the  questionnaires  were  received  from  the  owners, 
the  A/E  firms  were  contacted  and  asked  to  fill  out  the  service  acquisition 
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questionnaire.  There  were  a  100  percent  response  rate  once  the  A/E  firms  knew 
that  they  were  referred  to  by  the  owners.  Table  5.1  characterizes  the  sample  used 
in  this  study. 


Table  5.1  Sample  Characterization 


SF 

Totals 

Location 

Source 

(1) 

(2) 

(3) 

(4) 

_ (5) _ 

(6) 

pool 

120,000 

$10,676,000 

Austin,  TX 

Local 

P002 

319,000 

$21,770,000 

Irving,  TX 

Manufactung 

P003 

197,000 

$19,000,000 

Travis  CA 

Retml 

P004 

62,957 

$4,104,725 

Laredo,  TX 

Hotel/Motel 

P005 

71,000 

$4,325,930 

Harris  C  TX 

Retail 

FW  Dodge 

P006 

65,000 

$11,600,000 

Plano,  TX 

School 

P007 

67,000 

$4,500,000 

Austin,  TX 

Local 

POOS 

78,000 

$6,070,658 

Austin,  TX 

School 

AVG 

122,495 

$10,255,914 

5.2  Characterization  of  QBS  and  Non  QBS 

The  Brooks  Act  is  very  clear  on  the  selection  process  for  A/E  firms.  The 
selection  of  the  firm  has  to  be  made  based  on  qualifications  alone,  then 
negotiations  occur  with  a  goal  of  obtaining  a  fair  and  reasonable  price  not  to 
exceed  six  percent  to  the  most  qualified  firm.  If  the  negotiations  are  unsuccessful, 
the  negotiation  process  goes  to  the  next  most  qualified  firm  and  so  on.  The  exact 
opposite  of  QBS  would  be  selection  based  on  price  alone  such  as  a  sealed  bid.  A 
third  category  is  a  combination  of  the  QBS  and  non-QBS.  This  can  happen  a 
number  of  ways,  but  typically,  the  owner  will  make  his  selection  after  reviewing 
pricing  data.  In  essence,  the  owner  is  looking  for  the  “Best  Value”.  Table  5.2 
below  again  summarizes  the  different  categories  that  were  addressed  earlier. 


Table  5.2  A/E  Selection  Methods  (Vanden  Bosch  et  al.  1996) 


DESCRIPTION 

(Price) 

Bid 

Price  is  the  only  selection  determinate. 

Combination 

(Pre-qualification,  Price) 

Two 

envelope 

The  first  envelope  contains  qualifications  only.  All  highly 
qualified  firms  are  pre-qualified.  Pre-qualified  firms  are  invited 
to  submit  price  proposals.  The  second  envelope  contains  the 
price  proposal.  Award  is  based  on  best  price. 

One 

(Best  value,  considering  qualifications  and  price) 

envelope 

Firms  simultaneously  submit  qualifications  and  price.  Award 
may  be  based  on  some  combination  of  price  and  qualifications. 

QBS 

The  first  envelope  contains  qualifications  only.  The  one  best 

Two 

qualified  firm  is  selected  and  asked  to  submit  a  price  proposal. 

envelope 

Price  is  negotiated  with  the  one  best  qualified  firm  only. 

Sole  source 

(Reputation  is  the  selection  determinant) 

The  firm  is  selected  based  on  their  reputation  and/or  familiarity 
with  client’s  project.  Price  may  or  may  not  be  negotiated. 

From  this  pilot  study,  only  two  categories  were  observed.  The  two  are 
“QBS”  and  “Combination”  based  on  responses  from  the  service  provider 
questionnaires.  Table  5.3  separates  these  two  types. 


5.3  QBS/non-QBS  Characterization 


QBS,  Q/Combination,  C 

(2) 

pool 

Combination 

P002 

Combination 

P005 

Combination 

P006 

Combination 

POOS 

QBS 

P004 

QBS 

P007 

QBS 

POOS 

QBS 

None  of  the  projects  from  the  sample  were  based  soley  on  price.  In  each 
case  some  degree  of  qualifications  was  a  factor.  The  key  to  separating  between 
“QBS”  and  “Combination”  was  question  number  three  in  the  Service  Acquisition 
Quesionnaire.  The  bottom  line  in  separating  the  two  was  if  the  A/E  firm  was 
selected  before  negotiations  took  place,  then  the  project  was  catogorized  as  QBS. 

Selection  of  these  sample  projects  was  not  done  on  a  purely  random  basis. 
The  projects  from  the  Texas  area  were  targeted  because  of  the  time  constraint  of 
this  effort.  However,  there  were  no  measures  taken  to  ensure  a  50  percent  sub- 
samples  existed  between  QBS  and  non-QBS  projects.  Of  the  projects  selected 
from  Texas  half  were  identified  as  QBS  and  the  other  half  as  a  combination  or 
“Best  Value”  which  is  explained  in  Table  5.2.  From  these  results,  the  conclusion 


is  that  price  pressures  are  probably  being  exerted  by  owners  on  A/E  firms  on  a 
large  number  of  projects  in  today’s  construction  industry. 

5.3  Analysis  of  Data 

There  are  numerous  ways  to  measure  project  success.  For  the  purpose  of 
this  study.  The  factors  will  be  broken  into  cost,  schedule,  changes,  positive  and 
negative  factors,  subjective  design  ranking  and  overall  project  success  ratings  by 
the  owners.  In  terms  of  cost,  the  area  of  concentration  will  be  percentage  of 
design  cost  and  various  cost  per  square  foot  comparisons.  Evaluation  of 
schedule  will  include  actual  time  and  also  the  subjective  opinions  of  the  owners. 
Changes  will  be  presented  as  a  percentage  of  actual  construction  cost.  Finally,  the 
positive  and  negative  factors  will  be  grouped  in  one  table  to  summarize  the 
owner’s  overall  ranking  of  the  success  of  the  project.  All  data  presented  will 
segregate  the  QBS  and  non-QBS  projects. 

5.4  Design  Cost 

The  first  analysis  is  to  look  at  the  cost  of  design.  Remember,  the  Brooks 
Act  allows  no  more  than  6  percent  of  estimated  construction  cost  for  Federal 
projects.  This  provides  a  check  on  whether  design  fees  exceed  the  six  percent 
statutory  limit  and  also  gives  a  relative  comparison  between  QBS  and  non-QBS. 
Columns  (3)  and  (4)  in  Table  5.4  represent  the  cost  of  design  and  construction  at 
the  completion  of  the  project.  Column  (5)  is  the  percentage  of  design  with  respect 
to  column  (4). 
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Table  5  4  Design  Cost  of  Sample  Projects 


Q/N 

Design,  D  $ 

Construction,  C  $ 

%D 

(2) 

(3) 

(4) 

pool 

C 

$300,000 

$10,676,000 

2.81 

P002 

C 

$1,400,000 

$21,770,000 

6.43 

P005 

c 

$126,000 

$4,325,930 

2.91 

P006 

c 

$610,000 

$11,600,000 

5.26 

C  Avg. 

$609,000 

$12,092,983 

4.35 

P003 

Q 

$1,386,000 

$19,000,000 

7.29 

P004 

Q 

$167,748 

$4,104,725 

4.09 

P007 

Q 

$4,500,000 

POOS 

Q 

$299,773 

$6,070,658 

4.94 

Q  Avg. 

$617,840 

$8,418,846 

5.44 

Focusing  on  column  (5)  in  Table  5.4,  two  percent  of  the  projects  exceeded 
the  6  percent  threshold  set  by  the  Brooks  Act,  one  from  each  sample.  The 
problem  with  evaluating  design  cost  is  the  various  definitions  that  exist  in  the 
construction  industry.  Design  fees  can  include  a  multitude  of  services  from  site 
visits  to  drawings  to  assistance  with  contract  administration.  The  Brooks  Act 
allows  no  more  than  6  percent  of  estimated  cost  of  construction,  however,  this 
only  includes  basic  services  such  as  providing  the  specifications  and  drawings  for 
the  project.  Other  services  such  as  site  visits  and  modification  support  can  be 
added  on  top  of  the  6  percent.  Therefore,  it  is  very  important  to  ensure  the 
design  services  for  each  project  are  specifically  defined. 

As  probably  expected,  the  owners  of  the  projects  that  applied  price 
pressures  on  the  A/E  firms  on  the  average  paid  20  percent  less  for  design  services. 
The  results  are  shown  in  column  (4)  of  Table  5.5. 


5.5  Cost/SF  Analysis 


Cost  per  square  foot  is  another  essential  success  factor  to  analyze. 
Designers,  developers  and  owners  are  continually  thinking  about  the  bottom  line, 
cost  per  square  foot.  Most  likely,  the  cost  of  the  building  or  leasing  space  is 
estimated  and  leased  on  cost  per  square  foot  basis.  Table  5.3  below  shows  several 
Cost/SF  comparisons.  Columns  (2)  and  (3)  are  basically  the  design  and 
constructions  costs  divided  by  the  total  square  feet  of  the  facility. 


Tab 

e  5.5  Cost/SF  for  the  Sample 

Const.  SF 

Q/N 

mm 

(2) 

■SB 

pool 

$2.50 

$88.97 

c 

P002 

$4.39 

$68.24 

c 

P005 

$1.77 

$60.93 

c 

P006 

$4.69 

$89.23 

c 

EB9 

$3.34 

$76.84 

P003 

$7.04 

$96.45 

Q 

P004 

$2.66 

$65.20 

Q 

P007 

NA 

$67.16 

Q 

P008 

$3.84 

$77.83 

Q 

BSSS 

$3.39 

$76.66 

*  Average  for  P003,  P004  and  POOS 


Analyzing  columns  (2)  and  (3)  from  Table  5.5,  the  owners  that  are 
applying  price  pressures  are  saving  1.5  percent  on  the  design  cost  and  spending  .23 
percent  more  on  the  construction  cost.  Using  the  average  size  facility  and 
construction  cost  from  columns  (2)  and  (3)  of  Table  5.2,  average  savings  of  the 
owners  on  design  costs  is  $6,125  and  the  increase  spending  for  construction  is 


# 


$22,049.  The  owners  that  went  through  a  qualifications  process  saved  more 
money  than  the  non-QBS  projects.  The  margin  is  essentially  the  same  for  a  sample 
size  of  eight  projects.  On  multi-million  dollar  projects,  a  difference  of  a  few 
thousand  dollars  is  not  statistically  significant. 

5.6  Schedule  Analysis 

Completing  a  project  on  time  is  extremely  important.  In  general,  the  less 
time  the  contractor  stays  on  the  project  the  more  profitable  he  will  be.  This  will 
also  give  the  owner  earlier  access  to  the  facility.  Regardless  which  side  of  the 
project  a  firm  is  on,  schedule  is  an  important  success  factor  to  measure.  In  table 
5.6,  colunms  (3)  and  (4)  show  planned  and  actual  duration  of  the  project  in 
months.  Column  (5)  is  the  percentage  difference  between  columns  (3)  and  (4). 


Tal 

3le  5.6  Schedule  Analysis 

f3B!!Bl 

Q/C 

Planned 

Actual 

%Difif. 

MBSm 

■SI 

KSH 

(4) 

(5)...  . 

pool 

C 

8 

8 

0 

P002 

c 

12 

13 

+  8 

P005 

c 

18 

18 

0 

P006 

c 

6 

6 

0 

+  2 

P003 

Q 

12 

24 

+  100 

P004 

Q 

8 

7 

-14 

P007 

Q 

16 

20 

+  25 

P008 

Q 

24 

24 

0 

Q  Avg: 

+  28 

The  non-QBS  type  projects  rated  much  better  in  terms  of  completing  on 
time.  Project  P003  seems  to  sway  the  results.  The  delays  in  that  project  were 
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mainly  caused  by  a  weather  delay  of  20  weeks  and  engineering  and  construction 
productivity  delays  of  18  weeks.  Even  if  that  data  point  is  thrown  out,  the  non- 
QBS  projects  would  have  still  completed  their  projects  10  percent  faster.  This  is 
opportunity  cost  lost  by  the  owner  and  the  contractor.  Again,  a  larger  sample  size 
and  a  more  specific  breakdown  of  delays  caused  solely  by  design  errors  should  be 
analyzed. 

5.7  Changes  Analysis 

The  majority  of  the  contractors  and  owners  prefer  no  changes  because  they 
disrupt  the  schedule  and  have  a  cascading  effect  on  other  aspects  of  the  project. 
Changes  occur  on  almost  every  project  and  the  key  is  to  minimize  the  changes  that 
are  controllable.  In  table  5.7,  column  (3)  is  the  number  of  changes  that  occurred 
on  the  project.  Column  (4)  represents  the  cost  of  the  changes  and  column  (5)  is 
the  percentage  of  changes  with  respect  to  the  actual  cost  of  construction. 


Tal 

jle  5.7  Changes  Analysis  for  the  Sample 

Q/C 

Changes  ($) 

mm 

(3) 

(4) 

(5) 

pool 

C 

25 

$  1,400,000 

13.1% 

P002 

c 

134 

$  2,000,000 

9.2% 

POOS 

c 

26 

$  1,751,036 

40.5% 

P006 

c 

0 

0 

0 

Avg; 

15.7% 

POOS 

Q 

108 

$  7,000,000 

36.8% 

P004 

Q 

4 

$  111,993 

2.7% 

P007 

Q 

NA 

$  20,000 

0.4% 

P008 

Q 

9 

$  35,000 

0.6% 

Avg: 

10.1% 
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The  standard  contingency  on  a  new  construction  project  is  typically  five 
percent.  These  data  show  that  the  QBS  process  appears  significantly  better  than 
the  non-QBS  for  this  small  sample.  The  QBS  projects  on  the  average  paid  5.6% 
less  for  changes.  This  seems  logical.  The  better  qualified  the  A/E  firm,  the  less 
design  errors  should  be  expected  during  a  project.  Naturally,  all  the  changes  were 
not  attributed  to  design  errors,  therefore  in  the  future,  the  questionnaire  should 
focus  mainly  on  design  changes. 

5.8  Owner’s  Subjective  Analysis 

The  final  analysis  is  the  subjective  project  perception  from  the  owner’s 
perspective.  Since  these  are  only  from  the  owners  perspective,  there  will  be  some 
bias.  In  fixture  studies,  perhaps  a  ranking  of  the  owner  and  project  can  be  done  by 
the  design  firm  as  well.  In  table  5.8,  column  (3)  are  the  negative  and  positive 
factors  that  influenced  the  project,  such  as  external,  owner  controlled  or  project 
specific  as  discussed  earlier  in  section  4.4.  See  appendix  A,  question  2.1.7  for  a 
complete  breakdown  of  these  factors.  Column  (4)  provides  the  design  ranking  on 
a  scale  of  1-5  with  1  being  very  unsuccessful  and  5  being  very  successful.  Column 
(5)  is  the  overall  ranking  of  the  project  from  0  to  100. 
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Table  5,8  Owner  Subjective  Analysis 


Project 

Q/C 

Factors  (-,+) 

(1) 

(2) 

_ (3) _ 

(4) 

(5) 

pool 

C 

3 

82 

P002 

C 

_ (°.  13) _ 

3 

80 

P005 

c 

5 

92 

P006 

c 

_ (iO) _ 

1 

90 

mi 

3 

86 

POOS 

Q 

3 

65 

P004 

Q 

_  (6.  3) 

4 

72 

P007 

Q 

(4,  6) 

4 

92 

POOS 

Q 

_ (3^8) _ 

4 

65 

Avg 

mMESM 

3.75 

74 

The  QBS  A/E  firms  ranked  higher  than  the  non-QBS  in  terms  of  design 
effectiveness,  however,  the  projects  from  the  non-QBS  ranked  higher  than  the 
QBS.  The  non-QBS  projects  did  have  more  positive  and  less  negative  factors  than 
the  QBS. 


5.9  Evaluation  of  Data  Collection  Method 

The  questionnaire  method  of  collecting  data  takes  a  long  time,  especially 
since  most  professionals  are  using  voice  mail  and  personal  contact  is  difficult. 
Initial  contact  can  take  three  to  four  calls.  Once  the  point  of  contact  is  located,  a 
high  degree  of  promoting  the  research  project  is  a  key  to  quick  response.  The 
response  rate  for  this  project  was  approximately;  25  percent  said  no;  25  percent 
returned  within  the  specified  time  stated  in  the  initial  request,  and  50  percent  took 
multiple  reminders.  Therefore,  it  is  very  important  in  the  initial  contact  to  be 


prepared.  In  short,  the  researcher  should  sound  enthusiastic,  search  for  common 
links,  be  concise,  and  get  a  firm  commitment. 

The  questionnaire  on  the  average  took  approximately  45  minutes  to 
complete  by  the  respondent.  Some  of  the  problems  that  occurred  during  the 
collection  of  data  can  be  improved  by  modifying  the  questionnaires.  The  author 
recommends  changing  the  following  questions: 


Project  Performance  Questionnaire 

a.  Current: 

2.1.5  What  is  the  approximate  size  or  design  capacity  of  this  facility? 
(EXAMPLES::  gross  square  feet,  number  of  beds,  etc. ) 


New: 

2.1.5  What  is  the  approximate  size  of  the  facility? _ SF 

Reason: 

Design  capacity  focuses  on  the  industrial  type  project. 

b.  Current: 

2.2. 1  What  was  the  capital  cost  breakdown,  by  the  following  major  cost 
categories,  for  the  estimated  cost  at  the  time  of  major  funding  authorization  and 
the  actual  final  cost  of  the  project?  In  order  to  assist  you  in  completing  this 
section,  guidelines  for  selected  cost  categories  follows: 

Owner  Costs:  The  direct  owner  incurred  costs,  excluding  procured  equipment  or 
any  subcontracts. 

Owner  Procured  Equipment/Materials:  The  costs  associated  with  owner 
procurement  of  any  equipment  or  materials  inclusive  of  ary  capitalized 
subcontract  costs  (i.e.,  procurement  by  a  subcontractor  on  an  owner's  purchase 
order). 


Total  Project  Cost 

Estimated  Cost 
* 

Actual  Cost 

Owner  Costs 

Owner  Procured  Equipment/Material 

Engineering  &  Design  Services 

Construction  Contractor  Equip,  Material 
&  Labor 

Commissioning,  Turnover,  &  Startup 

Contingency 

xxxxxxxxx 

X 

Other 

Total  Project  Cost 

*  at  Authorization/ Appropriation 


New; 

2.1.1  What  is  the  cost  breakdown  of  the  project; 

Estimated  Actual 


a.  Construction  _ 

b.  Land/Capital  _ 

c.  Design  _ 

Describe  the  design  services  that  are  being  provided: 


d.  Other 


e.  Total  Cost: _ 

Reason: 

Some  of  the  factors  in  the  original  question  are  more  for  industrial  type 
projects.  Also,  the  description  of  the  design  is  necessary  to  ensure  the  research  is 
comparing  the  same  amount  of  services.  For  example,  if  one  design  firm  is  only 
providing  drawings  while  the  other  is  providing  additional  site  visits,  etc.,  then  it  is 
not  fair  to  compare  the  two. 


c.  Current: 


2.4. 1 .  What  was  the  total  approximate  number  of  change  orders  issued  (including 
engineering  and  construction)! 


2.4.2.  What  was  the  total  dollar  amount  of  all  change  orders? 

$ _ 

(approximate,  if  necessary). 

2.4.3  What  was  the  net  change  in  the  completion  date  resulting  from  change 
orders? _ ^months 

Did  the  net  impact  of  the  changes  increase  or  decrease  the  length  of  the  original 
project  duration?  (Please  check  only  one  answer) 

[  ]  Increase  [  ]  Decrease  [  ]  No  change  in  project  duration 


New: 

2.4. 1  What  was  the  total  approximate  number  and  cost  of  change  orders  issued: 

Number  Cost 


Due  to  Design  Errors 
Due  to  Unforeseen  conditions 
Due  to  Owner 
Other _ 


Total 


2.4.  2  Omit 

2.4.3  What  was  the  net  change  in  the  completion  date  resulting  from  change 
orders? _ ^months.  [  ]  increase  [  ]  decrease 


Reason; 


Data  is  in  a  more  useful  format.  Also,  it  can  specifically  isolate  the  design 
category  which  is  the  focus  of  the  study. 

d.  Current: 

2.5.1  What  percent  of  design  capacity  (usable  space,  or  capacity)  was  planned  or 
anticipated  at  the  time  this  project  was  authorized  and  actually  and  actually 
obtained  6  months  after  substantial  completion? 

2.5.2  What  percent  of  faciltiy  utilization  was  planned  or  anticipated  (at  the  time 
this  project  was  authorized)  and  actually  obtained  at  6  months  after  the  end  of 
startup? 


Planned  Obtained 

a.  6  months  %  _ 


b.  If  planned  utilization  differed  from  that  actually  obtained,  what  were  the  main 
reasons  for  the  differrence?  (Please  check  all  appropriate  box(es)) 


Reason 

Reason 

[  ] 

Availability  of  facility 

[  ]  Maintainability 

[  ] 

Quality 

[  ]  Availability  of  users 

[  ] 

Performance 

[  ]  Market  Demand 

[  ] 

Other  (please  specifV) 

2.5.3  Have  there  been  any  unanticipated  renovations  since  substantial  completion? 
(please  check  only  one  answer) 

[  ]  No  -  (If  no,  please  skip  to  the  next  question  below) 

[  ]  Yes  -  (If yes,  please  answer  the  following  questions,  3a,  3b,  3c). 

a.  What  was  the  cost  of  the  renovations?  $ _ 

b.  How  long  did  the  renovations  take?  _ months 


c.  Why  were  these  renovations  needed?  {Please  check  all  that  apply) 


Reason 

Reason 

[  ] 

Facility  Function 
Modification 

[  ] 

Regulatory  Change 

[  ] 

OperabiUty 

[  ] 

Design  Error 

[  ] 

Maintainability 

[  ] 

Increase  Capacity 

[  ] 

Construction  Error 

[  ] 

Other  {Please  specify) 

New: 

2.5.  1  -  2.5.3  Omit 
Reason: 

Questions  in  this  section  pertain  to  industrial  type  projects. 

e.  Current: 

2.7.2  We  would  like  to  know  your  overall  opinion  of  how  well  the  design 
professionals  performed  on  this  project,  taking  into  consideration  all  the  areas  that 
we  have  just  covered.  On  a  scale  of  1  to  5,  with  1  being  very  unsuccessful  and  5 
being  very  successful,  please  provide  us  with  your  rating  of  the  overall  effort. 

1  2  3  4  5 

What  are  your  main  reasons  for  your  assessment  of  the  design  professional? 

New: 

2.7.2  We  would  like  to  know  your  overall  opinion  of  how  well  the  design 
professionals  performed  on  this  project,  taking  into  consideration  all  the  areas  that 
we  have  just  covered.  On  a  scale  of  1  to  10,  with  1  being  very  unsuccessful  and 
10  being  very  successful,  please  provide  us  with  your  rating  of  the  overall  effort. 

123456789  10 
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What  are  your  main  reasons  for  your  assessment  of  the  design  professional? 


Reason: 

Need  to  have  more  variance  in  the  scale.  Most  ratings  were  3  or  4. 

Service  Acquisition  Questionnaire 

a.  Current: 

3.  Check  the  one  statement  that  best  describes  the  timing  of  the  cost  proposal 
submission  by  the  service  provider  to  the  owner: 

[  ]  Submitted  as  a  bid,  with  no  pre-qualification. 

[  ]  Submitted  concurrently  with  firm’s  qualifications. 

[  ]  Submitted  after  pre-qualifying,  but  prior  to  selection. 

[  ]  Submitted  after  pre-qualifying,  and  after  selection.. 

[  ]  Other  (Please  specify) 


New: 

3.  Check  the  one  statement  that  best  describes  the  timing  of  the  cost  proposal 
submission  by  the  service  provider  to  the  owner: 

[  ]  Submitted  as  a  bid,  with  no  pre-qualification. 

[  ]  Submitted  concurrently  with  firm’s  qualifications. 

[  ]  Submitted  after  pre-qualifying,  but  prior  to  selection. 

[  ]  Submitted  after  pre-qualifying,  and  after  selection. 

but  prior  to  contract  award/negotiations. 

[  ]  Other  (Please  specify) 


Reason: 


There  needs  to  be  a  distinction  between  contract  award  and  selection.  This  is  the 
premise  that  this  study  is  based  on.  Does  the  A/E  firm  get  selected  to  negotiate 
before  the  owner  sees  any  pricing  data?  If  the  answer  is  yes  then  it  is  non-QBS.  If 
the  answer  is  no  then  it  is  QBS. 


Chapter  6:  Conclusions  and  Policy  Implications 

6.1  Conclusions 

Why  is  it  necessary  to  always  pick  the  best  design  firm?  If  a  firm  is 
professionally  qualified  for  a  specific  project,  then  is  it  qualified  to  do  the  job?  The 
“Best  Value”  method  seems  to  be  widely  used  within  the  construction  industry. 
This  makes  common  sense  and  is  the  way  most  consumers  think,  whether  it  be 
choosing  a  service  or  buying  a  product.  People  will  normally  choose  their  services 
based  on  quality  but  within  their  budget.  The  incremental  value  of  quality  service 
must  be  equal  to  the  incremental  cost  associated  vwth  paying  more  for  that  benefit. 
At  the  same  time,  a  marginal  performer  just  above  the  qualification  line  is  not  the 
optimum  solution  either. 

The  Brooks  Act  was  passed  over  25  years  ago  with  the  philosophy  of 
choosing  the  best  first  and  then  negotiating  the  price  with  a  6  percent  statutory 
limitation.  The  design  costs  are  broken  into  two  parts  which  included  the  initial 
design  fees  to  produce  the  drawing  and  specifications  and  then  the  engineering 
field  services.  The  combination  of  these  two  can  easily  exceed  6  percent. 

It  is  time  to  take  another  look  at  the  Brooks  Act,  and  the  evaluators  must 
use  statistical  data  to  back  up  any  decisions.  There  is  a  new  era  evolving  in  the 
design  industry,  as  A/E  firms  fi'om  other  countries  are  entering  the  US  market  and 
are  able  to  compete  quickly  because  they  are  producing  the  same  quality  of 
drawings  for  less  money.  That  is  probably  one  of  the  reasons  why  much  of  the 
private  industry  is  beginning  to  see  a  shift  towards  “best  value”  engineering 
acquisition  method. 


Realizing  that  this  is  only  a  pilot  study  of  eight  projects  and  a  more  in- 
depth  investigation  with  more  projects  picked  at  random  should  be  conducted,  the 
following  conclusions  were  apparent  from  this  limited  sample: 


•  Half  of  the  projects  selected  were  non-QBS,  and  none  were 
competitive  low  bid.  However,  several  used  price  and  qualification 
to  select. 

•  The  percentage  of  design  cost  of  non-QBS  projects  are  20  percent 
less  than  QBS  projects. 

•  There  was  no  significant  difference  in  cost  per  square  foot  between 
QBS  and  non-QBS,  including  design  and  construction  cost. 

•  The  non-QBS  projects  completed  26  percent  more  on  schedule 
than  the  QBS  projects. 

•  The  QBS  projects  experienced  less  design  changes  than 
non-QBS. 

•  There  was  no  direct  correlation  with  design  ranking  and  overall 
project  performance. 

•  Both  questionnaires  need  to  be  modified  for  future  studies  to  focus 
more  on  design  issues  and  to  simplify  the  questions.  Suggestions 
are  given  in  this  thesis. 

The  null  hypothesis  for  this  study  is  that  there  is  no  significant  relationship 
between  project  performance  and  the  method  of  selection  of  the  design  service 
provider.  Only  a  few  factors  were  looked  at  in  this  study  and  the  study  is 
inconclusive  although  it  is  intriguing  at  the  same  time.  This  pilot  study  is  a  good 
indicator  of  a  possible  change  that  is  occurring  within  the  private  sector. 


Specifically,  this  was  noticeable  by  observing  the  results  of  a  small  sample  of 
owners  hiring  A/E  firms.  However,  there  is  no  clear  solution  at  this  point  and  a 
solid  study  researching  the  benefits  and  shortfalls  of  A/E  acquisition  is  warranted. 

6.2  Policy  Implications 

Why  continue  with  business  as  usual  when  the  times  are  quickly  changing? 
As  mentioned  earlier,  there  have  been  improvements  to  the  FAR  with  the  Federal 
Streamlining  Act.  Also  Design/Build  is  another  example  of  how  the  design 
selection  method  is  transitioning.  The  Government  spends  millions  of  dollars  on 
outsourcing  A/E  firms.  Why  not  take  a  thorough  look  at  the  Brooks  Act  and 
address  the  applicability  of  moving  into  the  21st  Century  with  the  current  system. 

The  overwhelming  policy  implication  is  that  there  is  no  clear  cut  solution 
and  there  should  not  be  one.  One  possibility  is  to  provide  a  system  where 
combination  of  qualifications  of  an  A/E  firm  and  complexity  of  the  project  should 
determine  who  is  selected.  As  the  complexity  of  the  project  increases  so  does  the 
screening  process  for  selecting  an  A/E  firm.  However,  cost  should  always  be  a 
factor.  This  brings  the  concept  back  of  choosing  the  “best  value.”  Not  only  does 
this  enhance  competition,  but  it  will  be  the  trend  in  the  private  construction 
industry  into  the  future.  Hopefully,  the  study  that  was  piloted  in  this  thesis  will  be 
performed  in  the  fiiture  and  provide  guidance  for  change. 
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Appendices 


Appendix  A:  Project  Performance  Questionnaire 


# 


# 


PROJECT  PERFORMANCE 
QUESTIONNAIRE 

Building  Version 


The  University  of  Texas  at  Austin 


15  Aug  96 


# 
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Project  Performance  Questionnaire 

Organization  Name:  _ _ _ _ 

Point  of  Contact: 

1.  Name: _ 

2.  Title: _ _ _ _ 

3.  Address:  _ _ _ _ 

4.  Tel  No:  _ Fax  No:  _ 

General  Project  Information: 

1.  Project  Name: _ 

(Is  the  above  name  for  the  project  correct?  If  not,  please  correct  it  on  the  line  below,) 


2.  In  what  town  or  city  is  the  project  located? 


In  what  state? _ 

3.  What  type  of  facility  is  this  project?  (Please  check  only  one) 


1  ]  Jail/Prison 

[  ]  Retail 

[  ]  School 

[  ]  State/County/City  government 

[  ]  Office  Building 
[  ]  Correctional  Facility 
[  ]  Maintenance  Facility 

[  ]  Other  (please  specify) 

4.  What  are  the  primary  products  or  services  produced  by  this  facility? _ 

5.  What  is  the  approximate  size  or  design  capacity  of  this  facility?  _ 

(EXAMPLES::  gross  square  feet,  number  of  beds,  etc, ) 

6.  Which  of  the  following  best  describes  the  site  on  which  this  project  was  built  ?  If  more 
than  25%  of  the  project  was  a  renovation,  please  classify  the  project  as  a 
Retrofit/Expansion.  (Please  check  only  one  answer) 

[  ]  New  construction  [  ]  Other  Q?lease  descnbe) 

[  ]  Retrofit/Expansion  _ _ 


7.  In  the  table  below,  circle  the  one  word  (negative,  none,  or  positive)  that  best  represents 
the  overall  influence  that  factor  had  on  the  project’s  performance.  After  checking  the  one 
best  word,  check  the  box  under  all  categories  ( time,  cost,  quality,  or  other)  affected  by 
that  factor.  Factors  left  blank  are  assumed  as  not  affecting  the  project. 

"Negative"  signifies  the  factor  negatively  affected  the  project ,  causing  time  delays,  increased  cost,  or  reduced 
quality.  "None"  signifies  the  factor  did  not  affect  the  project,  and  had  no  influence  on  time,  cost  or  quality. 
"Positive  "  signifies  the  factor  positively  affected  the  project,  causing  time  acceleration,  reduced  costs,  or 
increased  quality. 


FACTOR 


Externally  driven  factors 


1.  competitive  const,  market 


2.  local  conditions 


3.  regulations  or  permits 


4.  labor  strikes 


5.  material  availabili 


6.  abnormal  weather 


7.  unrest  insurrection,  war 


8.  acts  of  god 


Owner  controlled  events 


re-proiect  plannin 


10.  contracting  strate 


11.  design  reviews 


12.  constructabili 


13.  construction  manager 


14.  time  constraints 


15.  use  of  incentives 


16.  financing  or  fimdin 


17.  partnerin 


Project  specific  factors 


18.  remoteness  of  the  site 


19.  complexity  of  project 


20.  new  technologv/desi 


21.  first  of  its  kind 


23. 


24.  hazardous  materials 


rience 


26.  proclivity  to  claim 


27.  attitude 


28.  safety  practices 


Other  (specify) _ 

29. 


Categories  Affected 

(Check  aQ  that  apply) 

Time 

1  Cost 

1  Oualitv 

Other 

□ 

□ 

□ 

□ 

0 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

0 

n 

0 

□ 

□ 

□ 

□ 

□ 

Time 

1  Cost 

Oualitv  lOther 

□ 

□ 

□ 

n 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

0 

□ 

□ 

□ 

□ 

□ 

n 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

Time 

1  Cost 

1  Oualitv 

Other 

□ 

□ 

n 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

n 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

Time  |  Cost  |  Quality  |  Other. 


□  □  □  □ 


EXPLANATION:  (If you  indicated  that  any  of  these  factors  affected  the  project,  please 
indicate  it  by  number  and  briefly  explain  below.  Use  the  back  of  the  page  if  necessary): 
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8,  Please  identify  the  prime  design  firm: 


Name  of  Firm: _ 

Address: _ _ _ 

Primary  Contact: 

Tide: _ 

Tel.  No:  _ _ _  Fax  No: _ _ 

2.2.  Cost  Information: 

1.  What  was  the  capital  cost  breakdown,  by  the  following  major  cost  categories,  for  the 
estimated  cost  at  the  time  of  major  funding  authorization  and  the  actual  final  cost  of  the 
project?  In  order  to  assist  you  in  completing  this  section,  guidelines  for  selected  cost 
categories  follows: 

Owner  Costs:  The  direct  owner  incurred  costs,  excluding  procured  equipment  or  any 
subcontracts. 


Owner  Procured  Eauipment/Materials:  The  costs  associated  with  owner  procurement 
of  any  equipment  or  materials  inclusive  of  any  capitalized  subcontract  costs  (i.  e., 
procurement  by  a  subcontractor  on  an  owner's  purchase  order). _ 


Total  Project  Cost 

Estimated  Cost  * 

Actual  Cost 

Owner  Costs 

Owner  Procured  Equipment/Material 

Engineering  &  Design  Services  i 

Construction  Contractor  Equip,  Material  & 
Labor 

Commissioning,  Turnover,  &  Startup 

Contingency 

xxxxxxxx 

XX 

Other 

Total  Project  Cost 

*  at  Authorization/Appropriation 


2, 


In  terms  of  construction  bidding,  what  were  the: 


Number  of  bidders: 

Low  bid: 

Second  low  bid: 

ffighbid: 

OR 

attach  the  construction  bid  list  for  the  project 


2.3.  Schedule  Information: 


1.  What  was  the  date  of  major  funding  authorization? _ 

2.  What  was  the  planned  duration  of  the  execution  schedule  (from  project  authorization 

to  substantial  completion)  at  project  authorization  (in  months)? _ months. 

3 .  What  was  the  actual  date  of  substantial  completion?  _ 

4.  If  there  were  any  schedule  overruns  greater  than  1  percent  of  total  project  duration, 
please  indicate  the  reason(s)  in  the  appropriate  box(es)  below  by  supplying  the  duration(s) 
of  the  change(s)  {Please  check  all  that  apply,) 


Delay  Weeks. 

[  ]  Scope/Design  Change  _ 

[  ]  Labor  Shortage  _ 

[  ]  Contract  Dispute  _ 

[  ]  Weather  _ 

[  ]  Strike  _ 

[  ]  Material  Shortage  /  Delivery  _ 


Delay  Weeks. 

[  ]  Funding  Change  _ 

[  ]  Regulatory  Change  _ 

[  ]  Equipment  Availability  _ 

[  ]  Construction  Productivity  _ 

[  ]  Engineering  Productivity  _ 

[  ]  Other  {please  specify)  _ 


5.  If  you  checked  “Scope/Design  Change”  in  4.,  above,  please  describe  the  change(s) 


2.4  Change  Information: 

1.  What  was  the  total  approximate  number  of  change  orders  issued  {including 
engineering  and 

construction)! _ 

2.  What  was  the  total  dollar  amount  of  all  change  orders? 

$ _ 

{approximate,  if  necessary). 

3 .  What  was  the  net  change  in  the  completion  date  resulting  from  change  orders? 

_ months 

Did  the  net  impact  of  the  changes  increase  or  decrease  the  length  of  the  original 
project  duration?  (Please  check  only  one  answer) 

[  ]  Increase  [  ]  Decrease  [  ]  No  change  in  project  duration 

4.  Were  there  any  individual  changes  after  project  authorization  that  exceeded  1  percent 
of  the  project  budget?  (Please  check  only  one  answer) 

[  ]No 

[  ]  Yes  -  {If”Yes*\  what  were  the  total  cumulative  effects  and  the  direction  of  these  changes 
on): 


a.  Cost  -  $  _ [  ]  Increase  or  [  ]  Decrease 

b.  Schedule  - _ months  [  ]  Increase  or  [  ]  Decrease 

c.  How  many  changes  were  1  percent  of  the  original  project  budget  or  greater? 

d.  What  were  the  reasons  for  the  changes?  (Please  check  all  that  apply) 


Reason  Amount 

(%) 

User  Change  - 

Schedule  Change  - 

Weather  - 

Differing  Site  - 

Conditions 

Estimating  Error  - 

Scope  &  Design  - 

Changes 


Reason  Amount(%) 

Funding  Change  _ 

Regulatory  Change  _ 

Strike  _ 

Design  Error  _ 

Market  Change  _ 

Other  (Please  _ 

specify) 


Operating  Information: 

1.  What  percent  of  design  capacity  (usable  space,  or  capacity)  was  planned  or  anticipated 
at  the  time  this  project  was  authorized  and  actually  obtained  6  months  after 
substantial  completion? 

Planned  Obtained 
6  months  - 


2.  What  percent  of  facility  utilization  was  planned  or  anticipated  (at  the  time  this  project 
was  authorized)  and  actually  obtained  at  6  months  after  the  end  of  startup? 

Plaimed  Obtained 
6  months  - ^  ^ 


b.  If  the  planned  utilization  differed  from  that  actually  obtained,  what  were  the  main 
reasons  for  the  cUfference?  (Please  check  all  appropriate  box(es)) 

Reason  Reason 

[  ]  Availability  of  facility  [  ]  Maintainability 

[  ]  Quality  (  ]  Availability  of  users 

[  ]  Performance  [  ]  Market  demand 

[  1  Other  (please  specify) _ 


3.  Have  there  been  any  unanticipated  renovations  since  substantial  completion?  (please 
check  only  one  answer) 

[  ]l^o  ~  (If  no,  please  skip  to  the  next  question  below,) 

[  ]  Yes  -  {If  yes,  please  answer  the  following  questions,  3a,  3b,  3c), 

a.  What  was  the  cost  of  the  renovations?  $ _ 

b.  How  long  did  the  renovations  take? _ months 

c.  Why  were  these  renovations  needed?  {Please  check  all  that  apply) 


Reason 

Reason 

[  ] 

Facility  Function 
Modification 

[  1 

Regulatory  Change 

[  ] 

Operability 

[  ] 

Design  Error 

[  ] 

Maintainability 

[  ] 

Increase  Capacity 

[  ] 

Construction  Error 

[  ] 

Other  (Please  specify) 

[  ] 

Improve  Efficiency 

2.6  Other  Information 


1.  Did  the  project  have  any  claims  that  required  arbitration,  litigation,  or  mediation? 
[  ]No~{If  no,  please  skip  to  the  next  question  below.) 

[  ]  Yes  -  {If  yes,  please  answer  the  following  question,  la). 

a.  If  so,  please  indicate  the  magnitude  and  provide  details. 


2.  Were  there  any  protests  regarding  selection  of  A/E's? 

[  ]No  -  {If  no,  please  skip  to  the  next  question  below.) 

[  ]  Yes  -  {If  yes,  please  answer  the  following  question,  2a). 

a.  If  so,  please  elaborate? 


2.7  Subjective  Evaluation  of  the  Project 

1.  The  following  questions  are  intended  to  subjectively  evaluate  the  facility.  ( Read  the  statement  about  the  project, 
then  provide  a  response  to  the  statement  on  a  scale  of  1  to  5,  with  J  meaning  you  strongly  disagree  with  the 
statement,  3  meaning  you  neither  agree  or  disagree  and  5  meaning  you  strongly  agree  with  the  statement  DK 
stands  for  ‘'Don't  Know' j. 


a. 


b. 


d. 

e. 

f 

g- 

h. 


j- 

k. 

l. 

m. 

n. 


In  general,  project  participants  worked  well  together.  1 

Examples  of  project  participants  are  owners,  customers,  design  contractors,  and 
consultants.  The  participants  worked  together  toward  the  common  goal  of 
successfully  designing  the  project 

Emphasis  was  placed  on  identifying  and  satisfying  the  needs  of  the  customer  by  the  1 
design  professional(s). 

This  project  was  characterized  by  high  quality,  professional  performance  by  project  1 
participants. 

Innovative  design  solutions  were  used  to  solve  project  problems  and  provide  a  state-  1 
of-the-art  facility 

Lessons  learned  from  previous  construction  projects  were  incorporated  into  this  1 
project  during  the  design  phase. 

The  facility  design  is  aesthetically  pleasing.  1 

The  goals  to  maintain  or  improve  the  quality  of  the  environment  were  met  or  1 
exceeded  by  this  project  (Goals  should  include  organization  and/or  regulatory) 

The  completed  facility  has  provided  a  safe  workplace  since  it  was  placed  into  1 
operations. 

The  goals  concerning  ease  of  facility  operation  were  met  or  exceeded.  {Examples  of  1 
ease  of  operation  goals  are  operating  staff  size  and  overtime.) 

This  project  met  or  exceeded  its  goals  concerning  the  number  of  days  it  was  available  1 
for  operation  in  a  year. 

There  has  been  little  need  for  any  major,  unplanned  facility  improvements  or  changes  1 
since  completion  of  this  project 

The  goals  concerning  the  ease  of  maintenance  were  achieved  by  the  execution  of  this  1 

project 

The  experience  of  the  design  professional  was  adequate  to  perform  the  design  of  this  1 
project 

The  design  professional  fostered  an  effective  level  of  communications  during  the  1 
course  of  the  design 


2  3  4  5 

2  3  4  5 

2  3  4  5 

2  3  4  5 

2  3  4  5 

2  3  4  5 

2  3  4  5 

2  3  4  5 

2  3  4  5 

2  3  4  5 

2  3  4  5 

2  3  4  5 

2  3  4  5 

2  3  4  5 


DK 

DK 

DK 

DK 

DK 

DK 

DK 

DK 

DK 

DK 

DK 

DK 

DK 

DK 


2.  We  would  like  to  know  your  overall  opinion  of  how  well  the  design  professionals 
performed  on  this  project,  taking  into  consideration  all  the  areas  that  we  have  just 
covered.  On  a  scale  of  1  to  5,  with  1  being  very  unsuccessful  and  5  being  very 
successful,  please  provide  us  with  your  rating  of  the  overall  effort. 


1  2  3  4  5 


What  are  your  main  reasons  for  your  assessment  of  the  design  professional? 


3.  In  ranking  this  project’s  success  against  all  projects  you  are  familiar  with,  in  what 
percentile  would  you  place  this  project?  (Circle,  or  "x”  the  appropriate  percentile  on  the 
scale  below.) 

Worst  Best 


0  5  10  15  20  25  30  35  40  45  50  55  60  65  70  75  80  85  90  95  100 
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Appendix  B:  Service  Acquisition  Questionnaire 


SERVICE  ACQUISITION 
QUESTIONNAIRE 


The  University  of  Texas  at  Austin 


Project  Questionnaire 


Project  Name: 

Prime  Design  Firm: _ _ 

Address: _ 

City: _ State _ Zip _ 

Primary  Contact:  _ 

Title: _ _ _ _ 

Teh  No:  _ Fax  No: _ 

Service  Provider  Selection  Factors 

1.  Check  the  one  statement  that  best  describes  the  service  provider’s  fee  for  this  project: 

[  ]  Fixed  price. 

[  ]  Reimbursable  costs  with  fixed  profit. 

[  ]  Reimbursable  costs  with  guaranteed  maximiun  price, 

[  ]  Reimbursable  costs  plus  incentive  payment. 

[  ]  Multiple  of  Hourly  Rate(s). 

[  ]  Percent  of  Construction. 

[  ]  Cost  plus  %  of  Cost 
[  ]  Other  (Please 

specify _ _ 

2.  Check  the  one  statement  that  best  describes  the  degree  of  cost  negotiation  pressure 
by  the  owner  on  the  service  provider: 

[  ]  None  [  ]  Some  [  ]  Intense  [  ]  Inflexible 

3.  Check  the  one  statement  that  best  describes  the  timing  of  the  cost  proposal 
submission  by  the  service  provider  to  the  owner: 

[  ]  Submitted  as  a  bid,  with  no  pre-qualification. 

[  ]. Submitted  concurrently  with  firm’s  qualifications. 

[  ]  Submitted  after  pre-qualifying,  but  prior  to  selection. 

[  ]  Submitted  after  pre-qualifying,  and  after  selection.. 

[  ]  Other  (Please  specify) 


4.  Check  the  one  statement  that  best  describes  the  format  of  the  cost  proposal: 
[  ]  Lump  sum. 

[  ]  Hourly  Rates  plus  multiplier. 

[  ]  Hours  (not  dollars)  provided. 

[  ]  Other  (Please  specify) 


5.  Check  the  one  statement  that  best  describes  the  degree  of  competition  among  the 
service  provider’s  seeking  to  be  selected  for  this  project: 


[  ]  Sole  source 

Pre — Qualified? 

[  ]  Yes  (  ]  No 

[  ]  2  or  3  competitors 

[  ]Yes 

[  ]No 

[  ]  4  or  5  competitors. 

[  lYes 

[  ]No 

[  ]  6  +  competitors 

[  ]  Other  (Please  specify) 

[  ]Yes 

[  ]No 

6.  Check  the  one  statement  that  best  describes  the  nature  of  the  pre-qualification,  if 
any,  that  was  required,  to  be  considered  for  this  project 

[  ]  No  pre-qualification  required. 

[  ]  by  owner’s  invitation  only. 

[  ]  open  to  any  interested  firm. 

[  ]  Other  (Please  specify) 


7.  Check  each  statement  below  that  describes  the  qualincation(s)  considered  in 
selecting  a  service  provider  for  this  project,  and  indicate  the  degree  of  importance  that  it 
played  in  the  selection: 


Importance:  Not 

[  ]  Technical  expertise  in  this  type  project.  1 
[  ]  Technical  expertise  in  similar  type 
projects.  1 

[  ]  Experience  in  this  type  of  project  1 
[  ]  Experience  in  similar  types  of  projects  1 
[  ]  Immediate  availability  to  begin  this 
project.  1 

[  ]  Other  (specify) _  1 


somewhat 

2  3  4 

2  3  4 

2  3  4 

2  3  4 

2  3  4 

2  3  4 


Very 

5 

5 

5 

5 

5 

5 


8.  On  what  primary  basis  was  the  service  provider  chosen? 

I  ]  Qualifications  [  ]  Cost  [  ]  Don’t  Know  [  ]  Other 

(specify) _ 
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